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2 Contents 


CHAPTER 1 


Overview 


1.1 Purpose 


Mamba is an open platform for the implementation and application of MCMC methods to perform Bayesian analysis 
in julia. The package provides a framework for (1) specification of hierarchical models through stated relationships 
between data, parameters, and statistical distributions; (2) block-updating of parameters with samplers provided, de- 
fined by the user, or available from other packages; (3) execution of sampling schemes; and (4) posterior inference. It 
is intended to give users access to all levels of the design and implementation of MCMC simulators to particularly aid 
in the development of new methods. 


Several software options are available for MCMC sampling of Bayesian models. Individuals who are primarily in- 
terested in data analysis, unconcerned with the details of MCMC, and have models that can be fit in JAGS, Stan, or 
OpenBUGS are encouraged to use those programs. Mamba is intended for individuals who wish to have access to 
lower-level MCMC tools, are knowledgeable of MCMC methodologies, and have experience, or wish to gain experi- 
ence, with their application. The package also provides stand-alone convergence diagnostics and posterior inference 
tools, which are essential for the analysis of MCMC output regardless of the software used to generate it. 


1.2 Features 


* An interactive and extensible interface. 
* Support for a wide range of model and distributional specifications. 


* An environment in which all interactions with the software are made through a single, interpreted programming 
language. 


— Any julia operator, function, type, or package can be used for model specification. 
— Custom distributions and samplers can be written in julia to extend the package. 

* Directed acyclic graph representations of models. 

* Arbitrary blocking of model parameters and designation of block-specific samplers. 

* Samplers that can be used with the included simulation engine or apart from it, including 


— adaptive Metropolis within Gibbs and multivariate Metropolis, 


approximate Bayesian computation, 
— binary, 


— Hamiltonian Monte Carlo (simple and No-U-Turn), 
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— simplex, and 


— slice samplers. 


Automatic parallel execution of parallel MCMC chains on multi-processor systems. 


Restarting of chains. 


Command-line access to all package functionality, including its simulation API. 


Convergence diagnostics: Gelman, Rubin, and Brooks; Geweke; Heidelberger and Welch; Raftery and Lewis. 


Posterior summaries: moments, quantiles, HPD, cross-covariance, autocorrelation, MCSE, ESS. 


Gadfly plotting: trace, density, running mean, autocorrelation. 


Importing of sampler output saved in the CODA file format. 


Run-time performance on par with compiled MCMC software. 


1.3 Getting Started 


The following julia command will install the package: 


julia» Pkg.add("Mamba") 


4 Chapter 1. Overview 


CHAPTER 2 


Contents 


2.1 Introduction 


2.1.1 MCMC Software 


Markov chain Monte Carlo (MCMC) methods are a class of algorithms for simulating autocorrelated draws from 
probability distributions //7/[26][37][74]. They are widely used to obtain empirical estimates for and make infer- 
ence on multidimensional distributions that often arise in Bayesian statistical modelling, computational physics, and 
computational biology. Because MCMC provides estimates of distributions of interest, and is not limited to point 
estimates and asymptotic standard errors, it facilitates wide ranges of inferences and provides for more realistic pre- 
diction errors. An MCMC algorithm can be devised for any probability model. Implementations of algorithms are 
computational in nature, with the resources needed to execute algorithms directly related to the dimensionality of their 
associated problems. Rapid increases in computing power and emergence of MCMC software have enabled models of 
increasing complexity to be fit. For all its advantages, MCMC is regarded as one of the most important developments 
and powerful tools in modern statistical computing. 


Several software programs provide Bayesian modelling via MCMC. Programs range from those designed for general 
model fitting to those for specific models. WinBUGS, its open-source incarnation OpenBUGS, and the ‘BUGS’ clone 
Just Another Gibbs Sampler (JAGS) are among the most widely used programs for general model fitting /54//68]. 
These three provide similar programming syntaxes with which users can specify statistical models by simply stating 
relationships between data, parameters, and statistical distributions. Once a model is specified, the programs auto- 
matically formulate an MCMC sampling scheme to simulate parameter values from their posterior distribution. All 
aforementioned tasks can be accomplished with minimal programming and without specific knowledge of MCMC 
methodology. Users who are adept at both and so inclined can write software modules to add new distributions and 
samplers to OpenBUGS and JAGS [90][92]. General model fitting is also available with the MCMC procedure found 
in the SAS/STAT ® software [46]. Stan is a newer and similar-in-scope program worth noting for its accessible syn- 
tax and automatically tuned Hamiltonian Monte Carlo sampling scheme /95]. PyMC is a Python-based program that 
allows all modelling tasks to be accomplished in its native language, and gives users more hands-on access to model 
and sampling scheme specifications /67]. Programs like GRIMS [63] and LaplacesDemon [96] represent another 
class of programs that fit general models. In their approaches, users work with the functional forms of (unnormalized) 
probability densities directly, rather a domain specific modelling language (DSL), for model specification. Examples 
of programs for specific models can be found in the R catalogue of packages. For instance, the arm package pro- 
vides Bayesian inference for generalized linear, ordered logistic or probit, and mixed-effects regression models /32/, 
MCMCpack fits a wide range of models commonly encountered in the social and behavioral sciences /57/, and many 
others that are more focused on specific classes of models can be found in the “Bayesian Inference” task view on the 
Comprehensive R Archive Network /66/. 
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2.1.2 The Mamba Package 


Mamba [84] is a julia /3/ package designed for general Bayesian model fitting via MCMC. Like OpenBUGS and 
JAGS, it supports a wide range of model and distributional specifications, and provides a syntax for model speci- 
fication. Unlike those two, and like PyMC, Mamba provides a unified environment in which all interactions with 
the software are made through a single, interpreted language. Any julia operator, function, type, or package can be 
used for model specification; and custom distributions and samplers can be written in julia to extend the package. 
Conversely, interactions with and extensions to OpenBUGS and JAGS can involve three different programming envi- 
ronments — R wrappers used to call the programs, their DSLs, and the underlying implementations in Component 
Pascal and C++. Advantages of a unified environment include more flexible model specification; tighter integration 
with supplied functions for convergence diagnostics and posterior inference; and faster development, testing, and de- 
bugging of extensions. Advantages of the BUGS DSLs include more concise model specification and facilitation of 
automated sampling scheme formulation. Indeed, sampling schemes must be selected manually in the initial release 
of Mamba. Nevertheless, Mamba holds other distinct advantages over existing offerings. In particular, it provides 
arbitrary blocking of model parameters and designation of block-specific samplers; samplers that can be used with 
the included simulation engine or apart from it; and command-line access to all package functionality, including its 
simulation API. Likewise, advantages of the julia language include its familiar syntax, focus on technical comput- 
ing, and benchmarks showing it to be one or more orders of magnitude faster than R and Python /2/. Finally, the 
intended audience for Mamba includes individuals interested in programming in julia; who wish to have low-level 
access to model design and implementation; and, in some cases, are able to derive full conditional distributions of 
model parameters (up to normalizing constants). 


Mamba allows for the implementation of an MCMC sampling scheme to simulate draws for a set of Bayesian model 
parameters (01,...,0,) from their joint posterior distribution. The package supports the general Gibbs /28]/33] 
scheme outlined in the algorithm below. In its implementation with the package, the user may specify blocks (O;) T 
of parameters and corresponding functions ( fj $a to sample each O), from its full conditional distribution p(O;|O V 
O;). Simulation performance (efficiency and runtime) can be affected greatly by the choice of blocking scheme and 
sampling functions. For some models, an optimal choice may not be obvious, and different choices may need to 
be tried to find one that gives a desired level of performance. This can be a time-consuming process. The Mamba 
package provides a set of julia types and method functions to facilitate the specification of different schemes and 
functions. Supported sampling functions include those provided by the package, user-defined functions, and functions 
from other packages; thus providing great flexibility with respect to sampling methods. Furthermore, a sampling 
engine is provided to save the user from having to implement tasks common to all MCMC simulators. Therefore, time 
and energy can be focused on implementation aspects that most directly affect performance. 


A summary of the steps involved in using the package to perform MCMC simulation for a Bayesian model is given 
below. 


1. Decide on names to use for julia objects that will represent the model data structures and parameters 


(01,...,0,). For instance, the Tutorial section describes a linear regression example in which predictor x 
and response y are represented by objects x and y, and regression parameters Bo, 31, and c? by objects bO, b1, 
and s2. 


2. Create a dictionary to store all structures considered to be fixed in the simulation; e.g., the 1ine dictionary in 
the regression example. 


3. Specify the model using the constructors described in the MCMC Types section, to create the following: 


(a) A Stochastic object for each model term that has a distributional specification. This includes param- 
eters and data, such as the regression parameters bO, b1, and s2 that have prior distributions and y that 
has a likelihood specification. 


(b) A vector of Sampler objects containing supplied, user-defined, or external functions 1 fj a for sam- 
pling each parameter block Oj. 


(c) A Model object from the resulting stochastic nodes and sampler vector. 


4. Simulate parameter values with the mcmc () function. 
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Input : Model prag O — (64 
Blocking (6;]; jia Such that UJ; e a on, 96; — 2. 
Functions ( f. y = such that f; samples from p(@ je \ 8j). 
Result: Simulated values {9°} from the joint distribution of ©. 
begin 
Ə + Initialize( ); 


for i = 1 to N do 
for j = 1 to B do 
| 9; + f;(0); 
end 
e + 0; 
end 
end 


Fig. 2.1: Mamba Gibbs sampling scheme 


5. Use the MCMC output to check convergence and perform posterior inference. 


2.2 Tutorial 


The complete source code for the examples contained in this tutorial can be obtained here. 


2.2.1 Bayesian Linear Regression Model 


In the sections that follow, the Bayesian simple linear regression example from the BUGS 0.5 manual /85] is used 
to illustrate features of the package. The example describes a regression relationship between observations x = 
(1,2,3,4,5)! and y = (1,3,3,3,5)' that can be expressed as 


y ~ Normal (p, o? T) 
w= XB 


with prior distribution specifications 


0 1000 0 
B ~ Normal (1. B H a= | 0 ion) 


c? ~ InverseGamma(a, = 0.001, 8, = 0.001), 


where 8 = (Bo, 81)', and X is a design matrix with an intercept vector of ones in the first column and g in the second. 
Primary interest lies in making inference about the £o, 81, and o? parameters, based on their posterior distribution. A 
computational consideration in this example is that inference cannot be obtain from the joint posterior directly because 
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of its nonstandard form, derived below up to a constant of proportionality. 


p(B,0°|y) « p(ylB, 07) p(B)p(o7) 


x exp { - 508 - Hr) Ez (8 - tn) (02) 77 exp (8/0?) 


A common alternative is to make approximate inference based on parameter values simulated from the posterior with 
MCMC methods. 


2.2.2 Model Specification 


Nodes 


In the Mamba package, terms that appear in the Bayesian model specification are referred to as nodes. Nodes are 
classified as one of three types: 


* Stochastic nodes are any model terms that have likelihood or prior distributional specifications. In the regression 
example, y, 8, and c? are stochastic nodes. 


* Logical nodes are terms, like jz, that are deterministic functions of other nodes. 
* Input nodes are any remaining model terms (X) and are considered to be fixed quantities in the analysis. 


Note that the y node has both a distributional specification and is a fixed quantity. It is designated a stochastic node 
in Mamba because of its distributional specification. This allows for the possibility of model terms with distributional 
specifications that are a mix of observed and unobserved elements, as in the case of missing values in response vectors. 


Model implementation begins by instantiating stochastic and logical nodes using the Mamba-supplied constructors 
Stochastic and Logical. Stochastic and logical nodes for a model are defined with a call to the Model con- 
structor. The model constructor formally defines and assigns names to the nodes. User-specified names are given on 
the left-hand sides of the arguments to which Logical and Stochastic nodes are passed. 


using Mamba 
## Model Specification 
model = Model ( 


y = Stochastic(1l, 
(mu, s2) -> MvNormal(mu, sqrt(s2)), 
false 

) , 


mu = Logical(l, 
(xmat, beta) -> xmat * beta, 
false 

) , 


beta = Stochastic(l, 


() -> MvNormal(2, sqrt (1000)) 


s2 = Stochastic ( 
() => InverseGamma(0.001, 0.001) 
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A single integer value for the first Stochastic constructor argument indicates that the node is an array of the 
specified dimension. Absence of an integer value implies a scalar node. The next argument is a function that may 
contain any valid julia code. Functions should be defined to take, as their arguments, the inputs upon which their nodes 
depend and, for stochastic nodes, return distribution objects or arrays of objects compatible with the Distributions 
package /7]. Such objects represent the nodes’ distributional specifications. An optional boolean argument after 
the function can be specified to indicate whether values of the node should be monitored (saved) during MCMC 
simulations (default: t rue). 


Stochastic functions must return a single distribution object that can accommodate the dimensionality of the node, or 
return an array of (univariate or multivariate) distribution objects of the same dimension as the node. Examples of 
alternative, but equivalent, prior distributional specifications for the bet a node are shown below. 


# Case 1: Multivariate Normal with independence covariance matrix 
beta = Stochastic(l, 
() -> MvNormal (2, sqrt (1000) ) 


# Case 2: One common univariate Normal 
beta = Stochastic(l, 
() -> Normal(0, sqrt (1000) ) 


# Case 3: Array of univariate Normals 
beta = Stochastic(l, 
() -> UnivariateDistribution[Normal(0, sqrt(1000)), Normal(0, sqrt(1000))] 


# Case 4: Array of univariate Normals 
beta = Stochastic(l, 
() -> UnivariateDistribution[Normal(0, sqrt(1000)) for i in 1:2] 


Case 1 is one of the multivariate normal distributions available in the Distributions package, and the specification 
used in the example model implementation. In Case 2, a single univariate normal distribution is specified to imply 
independent priors of the same type for all elements of beta. Cases 3 and 4 explicitly specify a univariate prior for 
each element of beta and allow for the possibility of differences among the priors. Both return arrays of Distribution 
objects, with the last case automating the specification of array elements. 


In summary, y and beta are stochastic vectors, s2 is a stochastic scalar, and mu is a logical vector. We note that 
the model could have been implemented without mu. It is included here primarily to illustrate use of a logical node. 
Finally, note that nodes y and mu are not being monitored. 


Sampling Schemes 


The package provides a flexible system for the specification of schemes to sample stochastic nodes. Arbitrary block- 
ing of nodes and designation of block-specific samplers is supported. Furthermore, block-updating of nodes can be 
performed with samplers provided, defined by the user, or available from other packages. Schemes are specified as 
vectors of Sampler objects. Constructors are provided for several popular sampling algorithms, including adaptive 
Metropolis, No-U-Turn (NUTS), and slice sampling. Example schemes are shown below. In the first one, NUTS is 
used for the sampling of beta and slice for s2. The two nodes are block together in the second scheme and sampled 
jointly with NUTS. 


## Hybrid No-U-Turn and Slice Sampling Scheme 
schemel = [NUTS(:beta), 
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Slice(:s2, 3.0)] 


## No-U-Turn Sampling Scheme 
scheme2 - [NUTS([:beta, :s2])] 


Additionally, users are free to create their own samplers with the generic Sampler constructor. This is particularly 
useful in settings were full conditional distributions are of standard forms for some nodes and can be sampled from 
directly. Such is the case for the full conditional of 8 which can be written as 


v(B|a?, y) x p(ylB, o?)»(8) 
1 T 
x epf (O 12 (8- 2). 
where X = (FAT + xs) and y = MX (X'y + Ee. and is recognizable as multivariate normal. 
Likewise, 
p(o^|B. y) x p(y|B, 07)p(o”) 


RA ep [E (Ss - XB)" (y x8) +84) ). 


whose form is inverse gamma with n/2+ o, shape and (y — X 3) ' (y — X B)/2-- Ba scale parameters. A user-defined 
sampling scheme to generate draws from these full conditionals is constructed below. Another example is given in the 
Pollution example for the sampling of multiple nodes. 


## User-Defined Samplers 


Gibbs_beta = Sampler([:beta], 


(beta, s2, xmat, y) -> 
begin 
beta mean - mean(beta.distr) 
beta invcov = invcov(beta.distr) 
Sigma = inv(xmat' * xmat / s2 + beta invcov) 
mu = Sigma « (xmat' x* y / s2 + beta invcov * beta mean) 


rand (MvNormal (mu, Sigma)) 
end 


Gibbs_s2 = Sampler([:s2], 
(mu, S2, y) -> 
begin 
a = length(y) / 2.0 + shape(s2.distr) 
b = sumabs2(y - mu) / 2.0 + scale(s2.distr) 
rand(InverseGamma(a, b)) 
end 
) 


## User-Defined Sampling Scheme 
scheme3 - [Gibbs beta, Gibbs. s2] 


In these samplers, the respective MvNormal(2, sqrt(1000)) and InverseGamma (0.001, 0.001) priors 
on stochastic nodes bet a and s2 are accessed directly through the distr fields. Features of the Distributions objects 
returned by beta.distrand s2.distr can, in turn, be extracted with method functions defined in that package or 
through their own fields. For instance, mean (beta.distr) andinvcov (beta.distr) apply method functions 
to extract the mean vector and inverse-covariance matrix of the beta prior. Whereas, shape (s2.distr) and 
scale(s2.distr) extract the shape and scale parameters from fields of the inverse-gamma prior. Distributions 
method functions can be found in that package's documentation; whereas, fields are found in the source code. 
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When possible to do so, direct sampling from full conditions is often preferred in practice because it tends to be more 
efficient than general-purpose algorithms. Schemes that mix the two approaches can be used if full conditionals are 
available for some model parameters but not for others. Once a sampling scheme is formulated in Mamba, it can be 
assigned to an existing model with a call to the set samplers! function. 


## Sampling Scheme Assignment 
setsamplers! (model, schemel) 


Alternatively, a predefined scheme can be passed in to the Mode1 constructor at the time of model implementation as 
the value to its samplers argument. 


2.2.3 Directed Acyclic Graphs 


One of the internal structures created by Model is a graph representation of the model nodes and their associations. 
Graphs are managed internally with the Graphs package [94]. Like OpenBUGS, JAGS, and other BUGS clones, 
Mamba fits models whose nodes form a directed acyclic graph (DAG). A DAG is a graph in which nodes are connected 
by directed edges and no node has a path that loops back to itself. With respect to statistical models, directed edges 
point from parent nodes to the child nodes that depend on them. Thus, a child node is independent of all others, given 
its parents. 


The DAG representation of a Model can be printed out at the command-line or saved to an external file in a format 
that can be displayed with the Graphviz software. 


## Graph Representation of Nodes 
>>> draw (model) 


digraph MambaModel { 
"mu" [shape="diamond", fillcolor="gray85", style="filled"]; 


"mu" => spits 

"xmat" [shape="box", fillcolor="gray85", style="filled"]; 
" xmat " => "mu na 

"beta" [shape-"ellipse"]; 
"beta" => Pius 


"s2" [shape="ellipse"]; 
"s2! => Wats 
"y" [shape="ellipse", fillcolor="gray85", style-"filled"]; 


>>> draw(model, filename-"lineDAG.dot") 


Either the printed or saved output can be passed manually to the Graphviz software to plot a visual representation of 
the model. If julia is being used with a front-end that supports in-line graphics, like /Julia [47], and the GraphViz 
julia package /24] is installed, then the following code will plot the graph automatically. 


using GraphViz 


>>> display (Graph (graph2dot (model))) 


A generated plot of the regression model graph is show in the figure below. 


Stochastic, logical, and input nodes are represented by ellipses, diamonds, and rectangles, respectively. Gray-colored 
nodes are ones designated as unmonitored in MCMC simulations. The DAG not only allows the user to visually check 
that the model specification is the intended one, but is also used internally to check that nodal relationships are acyclic. 
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Fig. 2.2: Directed acyclic graph representation of the example regression model nodes. 


2.2.4 MCMC Simulation 
Data 


For the example, observations (a, y) are stored in a julia dictionary defined in the code block below. Included are 
predictor and response vectors : x and : y as well as a design matrix : xmat corresponding to the model matrix X. 


## Data 

line = Dict{Symbol, Any} ( 
ix => 2 3, 4, 5], 
¿y => [1, 3, 3, 3, 5] 

) 

line[:xmat] = [ones(5) line[:x]] 


Initial Values 


A julia vector of dictionaries containing initial values for all stochastic nodes must be created. The dictionary keys 
should match the node names, and their values should be vectors whose elements are the same type of structures as 
the nodes. Three sets of initial values for the regression example are created in with the following code. 


## Initial Values 
inits = [ 
Dict{Symbol, Any} ( 
fy => Lline[:yl, 
:beta => rand(Normal (0, 1), 2), 
:s2 => rand(Gamma(1, 1)) 
) 
for 1 in 1:3 


] 


12 Chapter 2. Contents 


Mamba.jl Documentation, Release 0.8.2 


Initial values for y are those in the observed response vector. Likewise, the node is not updated in the sampling 
schemes defined earlier and thus retains its initial values throughout MCMC simulations. Initial values are generated 
for beta from a normal distribution and for s2 from a gamma distribution. 


MCMC Engine 


MCMC simulation of draws from the posterior distribution of a declared set of model nodes and sampling scheme is 
performed with the mcmc () function. As shown below, the first three arguments are a Model object, a dictionary of 
values for input nodes, and a dictionary vector of initial values. The number of draws to generate in each simulation 
run is given as the fourth argument. The remaining arguments are named such that burnin is the number of initial 
values to discard to allow for convergence; thin defines the interval between draws to be retained in the output; and 
chains specifies the number of times to run the simulator. Results are retuned as Chains objects on which methods 
for posterior inference are defined. 


## MCMC Simulations 


setsamplers! (model, schemel) 
siml = mcmc(model, line, inits, 10000, burnin-250, thin-2, chains-3) 


setsamplers! (model, scheme2) 
sim2 = mcmc(model, line, inits, 10000, burnin=250, thin=2, chains=3) 


setsamplers! (model, scheme3) 
sim3 = mcmc(model, line, inits, 10000, burnin=250, thin-2, chains=3) 


Parallel Computing 


The simulation of multiple chains will be executed in parallel automatically if julia is running in multiprocessor mode 
on a multiprocessor system. Multiprocessor mode can be started with the command line argument julia -p n, 
where n is the number of available processors. See the julia documentation on parallel computing for details. 


2.2.5 Posterior Inference 


Convergence Diagnostics 


Checks of MCMC output should be performed to assess convergence of simulated draws to the posterior distribution. 
Checks can be performed with a variety of methods. The diagnostic of Gelman, Rubin, and Brooks /3/]/10] is one 
method for assessing convergence of posterior mean estimates. Values of the diagnostic's potential scale reduction 
factor (PSRF) that are close to one suggest convergence. As a rule-of-thumb, convergence is rejected if the 97.5 
percentile of a PSRF is greater than 1.2. 


>>> gelmandiag(siml, mpsrf-true, transform-true) |» showall 


Gelman, Rubin, and Brooks Diagnostic: 


PSRF 97.5% 

beta[1] 1.009 1.010 
beta[2] 1.009 1.010 

s2 1.008 1.016 
Multivariate 1.006 NaN 


The diagnostic of Geweke /35/ tests for non-convergence of posterior mean estimates. It provides chain-specific test 
p-values. Convergence is rejected for significant p-values, like those obtained for s2. 
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>>> gewekediag(siml) |» showall 


Geweke Diagnostic: 
First Window Fraction = 0.1 


Second Window Fraction = 0.5 


Z-score p-value 


beta[1] 1,237 0.2162 
beta[2] -1.568 0.1168 
s2 1.710 0.0872 
Z-score p-value 

beta[1] -1.457 0.1452 
beta[2] 1.752. 10450797 
s2 -1.428 0.1534 
Z-score p-value 

beta[1] 0.550 0.5824 
beta[2] -0.440 0.6597 
s2 0.583 0.5596 


The diagnostic of Heidelberger and Welch /[44] tests for non-convergence (non-stationarity) and whether ratios of 
estimation interval halfwidths to means are within a target ratio. Stationarity is rejected (0) for significant test p-values. 
Halfwidth tests are rejected (0) if observed ratios are greater than the target, as is the case for s2 and beta [1]. 


>>> heideldiag(siml) |» showall 


Heidelberger and Welch Diagnostic: 


Target Halfwidth Ratio = 0.1 
Alpha = 0.05 
Burn-in Stationarity p-value Mean Halfwidth Test 
beta[1] 251 0.0680 0.57366275 0.053311283 d 
beta[2] 738 0.0677 0.81285744 0.015404173 1 
s2 738 0.0700 1.00825202 0.094300432 1 
Burn-in Stationarity p-value Mean Halfwidth Test 
beta[1] 25 0.1356 0.6293320 0.065092099 0 
beta[2] 251 1 0.0711 07934633 0.019215278 il 
s2 251 0.4435 1.4635400 0.588158612 0 
Burn-in Stationarity p-value Mean Halfwidth Test 
beta[1] 251 0.0515 0.5883602 0.058928034 0 
beta[2] 1225 0.1479 0.8086080 0.018478999 1 
s2 254 0.6664 0.9942853 0.127959523 0 


The diagnostic of Raftery and Lewis /71][72] is used to determine the number of iterations required to estimate a 
specified quantile within a desired degree of accuracy. For example, below are required total numbers of iterations, 
numbers to discard as burn-in sequences, and thinning intervals for estimating 0.025 quantiles so that their estimated 
cumulative probabilities are within 0.025+0.005 with probability 0.95. 


>>> rafterydiag(siml) |» showall 


Raftery and Lewis Diagnostic: 


Quantile (q) = 0.025 

Accuracy (r) = 0.005 

Probability (s) = 0.95 
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Thinning Burn-in Total Nmin Dependence Factor 

beta[1] 2 267 17897 3746 4.7776295 
beta[2] 2 267 17897 3746 4.7776295 
s2 2 257 8689 3746 2.3195408 
Thinning Burn-in Total Nmin Dependence Factor 

beta[1] 4 271 2.1759x104 3746 5.8085958 
beta[2] 4 275 2.8795x104 3746 7.6868660 
s2 2 257 8.3450x103 3746 22211096 
Thinning Burn-in Total Nmin Dependence Factor 

beta[1] 2 269 2.0647x104 3746 5451057459 
beta[2] 2 263 1.4523x104 3746 3.8769354 
s2 2 255 7.8770x103 3746 2.1027 763 


More information on the diagnostic functions can be found in the Convergence Diagnostics section. 


Posterior Summaries 


Once convergence has been assessed, sample statistics may be computed on the MCMC output to estimate features of 
the posterior distribution. The StatsBase package /50] is utilized in the calculation of many posterior estimates. Some 
of the available posterior summaries are illustrated in the code block below. 


## Summary Statistics 
>>> describe (sim1) 


Iterations = 252:10000 
Thinning interval = 2 
Chains = 1,2,3 

Samples per chain = 4875 


Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
0.5971183 1.14894446 0.0095006014 0.016925598 4607.9743 
0.8017036 0.34632566 0.0028637608 0.004793345 4875.0000 
1.2203777 2.00876760 0.0166104638 0.101798287 389.3843 


Quantiles: 
25.595 250% 50.0% 75.0% OF 5% 
beta[1] -1.74343373 0.026573102 0.59122696 1.1878720 2.8308472 
beta[2] 0.12168742 0.628297573 0.80357822 0.9719441 1.5051573 
s2 0417091385 0.383671702 0.65371989 1.2206381 6.0313970 


## Highest Posterior Density Intervals 
>>> hpd(siml) 


95% Lower 95% Upper 

beta[1] -1.75436235 2.8109571 
beta[2] 0.09721501 1.4733163 
s2 0.08338409 3.8706865 


## Cross-Correlations 
>>> cor(siml) 


beta[1] beta[2] s2 
beta[1] 1.000000000 -0.905245029 0.027467317 
beta[2] -0.905245029 1.000000000 -0.024489462 
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s2 0.027467317 -0.024489462 1.000000000 


## Lag-Autocorrelations 


>>> autocor (sim1) 


Lag 2 Lag 10 Lag 20 Lag 100 
beta[1] 0.24521566 -0.021411797 -0.0077424153 -0.044989417 
beta[2] 0.20402485 -0.019107846 0.0033980453 -0.053869216 
s2 0.85931351 0.568056917 0.3248136852 0.024157524 

Lag 2 Lag 10 Lag 20 Lag 100 
beta[1] 0.28180489 -0.031007672 0.03930888 0.0394895028 
beta[2] 0.25905976 -0.017946010 0.03613043 0.0227758214 
s2 0.92905843 0.761339226 0.58455868 0.0050215824 

Lag 2 Lag 10 Lag 20 Lag 100 
beta[1] 0.38634357 -0.0029361782 -0.032310111 0.0028806786 
beta[2] 0.32822879 -0.0056670786 -0.020100663 -0.0062622517 
s2 0.68812720 0.2420402859 0.080495078 -0.0290205896 


## State Space Change Rate (per Iteration) 
>>> changerate (siml) 


Change Rate 


beta[1] 0.844 
beta[2] 0.844 

s2 1.000 
Multivariate 1.000 


## Deviance Information Criterion 
>>> dic(siml) 


DIC Effective Parameters 
pD 13.828540 122661193 
pV 22.624104 9.963 9015 


Output Subsetting 


Additionally, sampler output can be subsetted to perform posterior inference on select iterations, parameters, and 
chains. 


## Subset Sampler Output 


>>> sim = sim1[1000:5000, ["beta[1]", "beta[2]"], :] 
>>> describe (sim) 
Iterations = 1000:5000 
Thinning interval = 2 
Chains = 1,2,3 
Samples per chain = 2001 
Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
beta[1] 0.62445845 1.0285709 0.013275474 0.023818436 1864.8416 
beta[2] 0.79392648 0.3096614 0.003996712 0.006516677 2001.0000 
Quantiles: 

25.59 25.0$ 50.0% 75.0% 97.5% 
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beta[1] -1.53050898 0.076745702 0.61120944 1.2174641 2.6906753 
beta[2] 0.18846617 0.618849048 0.79323126 0.9619767 1.4502109 


File /O 


For cases in which it is desirable to store sampler output in external files for processing in future julia sessions, read 
and write methods are provided. 


## Write to and Read from an External File 
write("siml.jls", siml) 
siml = read("siml.jls", ModelChains) 


Restarting the Sampler 


Convergence diagnostics or posterior summaries may indicate that additional draws from the posterior are needed for 
inference. In such cases, the mcmc () function can be used to restart the sampler with previously generated output, as 
illustrated below. 


## Restart the Sampler 
>>> sim = mcmc (sim1, 5000) 
>>> describe (sim) 


Iterations = 252:15000 
Thinning interval = 2 
Chains = 1,2,3 

Samples per chain = 7375 


Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
beta[1] 0.59655228 1.1225920 0.0075471034 0.014053505 6380.79199 
beta[2] 0.80144540 0.3395731 0.0022829250 0.003954871 7372.28048 
s2 1.18366563 1.8163096 0.0122109158 0.070481708 664.08995 


Quantiles: 
2.5% 254.05 50.0% 75.0% 97.5% 
beta[1] -1.70512374 0.031582137 0.58989089 1.1783924 2.8253668 
beta[2] 0.12399073 0.630638800 0.80358526 0.9703569 1.4939817 
s2- 0.17075261 0.382963160 0.65372440 1.2210168 5.7449800 


Plotting 


Plotting of sampler output in Mamba is based on the Gadfly package [45]. Summary plots can be created and written 
to files using the plot and draw functions. 


## Default summary plot (trace and density plots) 
p = plot(siml) 


## Write plot to file 
draw(p, filename-"summaryplot.svg") 


The plot function can also be used to make autocorrelation and running means plots. Legends can be added with the 
optional legend argument. Arrays of plots can be created and passed to the draw function. Use nrow and ncol to 
determine how many rows and columns of plots to include in each drawing. 
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Fig. 2.3: Trace and density plots. 
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## Autocorrelation and running mean plots 
p = plot(siml, [:autocor, :mean], legend=true) 
draw(p, nrow=3, ncol=2, filename="autocormeanplot.svg") 
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Fig. 2.4: Autocorrelation and running mean plots. 


## Pairwise contour plots 
p = plot(siml, :contour) 
draw(p, nrow=2, ncol=2, filename="contourplot.svg") 
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2.2.6 Computational Performance 


Computing runtimes were recorded for different sampling algorithms applied to the regression example. Runs wer 
performed on a desktop computer with an Intel 15-2500 CPU @ 3.30GHz. Results are summarized in the table below. 
Note that these are only intended to measure the raw computing performance of the package, and do not account for 
different efficiencies in output generated by the sampling algorithms. 


Table 2.1: Number of draws per second for select sampling algorithms in Mamba. 


Adaptive Metropolis Slice 
Within Gibbs | Multivariate | Gibbs | NUTS | Within Gibbs | Multivariate 
16,700 11,100 27,300 | 2,600 13,600 17,600 


2.2.7 Development and Testing 


Command-line access is provided for all package functionality to aid in the development and testing of models. Ex- 
amples of available functions are shown in the code block below. Documentation for these and other related functions 
can be found in the MCMC Types section. 


## Development and Testing 


setinputs! (model, line) # Set input node values 
setinits! (model, inits[1]) # Set initial values 
setsamplers! (model, schemel) # Set sampling scheme 

showall (model) # Show detailed node information 
logpdf (model, 1) # Log-density sum for block 1 
logpdf (model, 2) # Block 2 

logpdf (model) # All blocks 

simulate! (model, 1) # Simulate draws for block 1 
simulate! (model, 2) # Block 2 

simulate! (model) # All blocks 


In this example, functions setinputs!, setinits!,and setsampler! allow the user to manually set the input 
node values, the initial values, and the sampling scheme form the model object, and would need to be called prior 
to logpdf and simulate!. Updated model objects should be returned when called; otherwise, a problem with 
the supplied values may exist. Method showall prints a detailed summary of all model nodes, their values, and 
attributes; logpdf sums the log-densities over nodes associated with a specified sampling block (second argument); 
and simulate! generates an MCMC draw for the nodes. Non-numeric results may indicate problems with distribu- 
tional specifications in the second case or with sampling functions in the last case. The block arguments are optional; 
and, if left unspecified, will cause the corresponding functions to be applied over all sampling blocks. This allows 
testing of some or all of the samplers. 


2.3 MCMC Types 


The MCMC types and their relationships are depicted below with a Unified Modelling Language (UML) diagram. In 
the diagram, types are represented with boxes that display their respective names in the top-most panels, and fields 
in the second panels. By convention, plus signs denote fields that are publicly accessible, which is always the case 
for these structures in julia. Hollow triangle arrows point to types that the originator extends. Solid diamond arrows 
indicate that a number of instances of the type being pointed to are contained in the originator. The undirected line 
between Sampler and Stochastic represents a bi-directional association. Numbers on the graph indicate that 


2.3. MCMC Types 21 


Mamba.jl Documentation, Release 0.8.2 


there is one (1), zero or more (0..*), or one or more (1..*) instances of a type at the corresponding end of a relationship. 


ScalarVariate 
+ value : Float64 
4 


Y. 


< 
: | Rea | 
AbstractVariate 


E DenseArray 
J 


ArrayVariate{N} ArrayStochastic{N} ScalarStochastic 
<] 


[^ 
+ value : Array{Float64, N} + distr : DistributionStruct + distr : UnivariateDistribution 
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Q P 


* value - 
; * symbol : Symbol AbstractStochastic 
AbstractLogical + monitor : Vector{Int} * distr 
+ eval: Function | *dstr | 


0.” + sources : Vector{Symbol} 
+ targets : Vector{Symbol} 


Sampler{T} 


+ params : Vector{Symbol} 
- + eval : Function 

+ nodes : Dict{Symbol, Any} +tune:T 

+ samplers : Vector{Sampler} + targets : Vectar[Symbal] 
+ states::Vector{ModelState} 


+ value : Array{Float64, 3} 
+ range : Range(Int) 


+ names : Vector{String} 
+ chains : Vector{Int} 


+ iter : Int 
+ burnin : Int R 
| ModelChains | delChai e + hasinputs : Bool - ModelState 
T Mog cad Bon + value : Vector{Float64} 
+ model: Model + tune : Vector{Any} 


Fig. 2.6: UML relational diagram of MCMC types and their fields. 


The relationships are as follows. Type Model contains a dictionary field (Dict (Symbol, Any)) of model nodes 
and a field (Vector {Sampler}) of one or more sampling functions. Nodes can be one of three types: 


* Stochastic nodes (ScalarStochastic or ArrayStochast ic) are any model terms that have 
likelihood or prior distributional specifications. 


* Logical nodes (ScalarLogical or ArrayLogical) are terms that are deterministic functions 
of other nodes. 


* Input nodes (not shown) are any other model terms and data types that are considered to be fixed 
quantities in the analysis. 


Stochastic and Logical are inherited from the Variate types and can be used with operators and in functions 
defined for that type. The sampling functions in Model each correspond to a block of one or more model parameters 
(stochastic nodes) to be sampled from a target distribution (e.g. full conditional) during the simulation. Finally, 
ModelChains stores simulation output for a given model. Detailed information about each type is provided in the 
subsequent sections. 


2.3.1 Variate 


ScalarVariate and ArrayVariate(N) are abstract types that serve as the basis for several concrete types in 
the Mamba package. Conceptually, they represent data structures that store numeric values simulated from target 
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distributions. Being abstract, these variate types cannot be instantiated and cannot have fields. They can, however, 
have method functions, which descendant subtypes will inherit. Such inheritance allows one to endow a core set 
of functionality to all subtypes by simply defining method functions once on the abstract types (see julia Types). 
Accordingly, a core set of functionality is defined for the variate types through the field and method functions discussed 
below. Although the (abstract) types do not have fields, their method functions assume that all subtypes will be declared 


with a value field. 


Declarations 


abstract ScalarVariate <: Real 
abstract ArrayVariate{N} <: DenseArray(Float64, N} 


typealias AbstractVariate Union{ScalarVariate, ArrayVariate} 
typealias VectorVariate ArrayVariate(íl) 
typealias MatrixVariate ArrayVariate{2} 


Type Hierarchy 


Subtypes of the variate types include the Dependent, Logical, Stochastic, and SamplerVariate types. 


DenseArray 


A 


ArrayVariate{N} 
+ value : Array{Float64, N} 


Y, V 


Matrix Vvariate 


+ value : ArrayfFloatB4, 2} 


ScalarVvariate 
+ value : Floatb4 


Fig. 2.7: UML relational diagram of Variate types and their fields. 


Field 


* value::T: scalar or array of Float 64 values that represent simulated values from a target distribution. 
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Methods 


Methods for ScalarVariate and ArrayVariate include mathematical operators, mathematical functions, and 
statistics defined in the base julia language for parent types Real and DenseArray. In addition, the following 
functions are provided. 


Function Description 
logit (x) log-odds 
invlogit (x) | inverse log-odds 


2.3.2 Dependent 


AbstractDependent is an abstract type designed to store values and attributes of model nodes, including param- 
eters 0,,...,0, to be simulated via MCMC, functions of the parameters, and likelihood specifications on observed 
data. It extends the base Variate types with method functions defined for the fields summarized below. Like the 
type it extends, values are stored in a value field and can be used with method functions that accept Float 64 or 
Array{Float64, N} type objects. 


Since parameter values in the Abst ractDependent structure are stored as a scalar or array, objects of this type 
can be created for model parameters of corresponding dimensions, with the choice between the two being user and 
application-specific. At one end of the spectrum, a model might be formulated in terms of parameters that are all 
scalars, with a separate instances of Abst ractDependent for each one. At the other end, a formulation might 
be made in terms of a single parameter array, with one corresponding instance of Abst ractDependent. Whether 
to formulate parameters as scalars or arrays will depend on the application at hand. Array formulations should be 
considered for parameters and data that have multivariate distributions, or are to be used as such in numeric operations 
and functions. In other cases, scalar parametrizations may be preferable. Situations in which parameter arrays are 
often used include the specification of regression coefficients and random effects. 


Declaration 


typealias AbstractDependent Union{AbstractLogical, AbstractStochastic} 


Fields 


* value::T: scalar or array of Float 64 values that represent samples from a target distribution. 
e symbol: : Symbol : identifying symbol for the node. 


e monitor: :Vector{Int} : indices identifying elements of the value field to include in monitored MCMC 
sampler output. 


e eval: :Function: function for updating the state of the node. 


e sources: :Vector {Symbol} : other nodes upon whom the values of this one depends. 


e targets: :Vector{Symbol} : Dependent nodes that depend on this one. Elements of targets are 
topologically sorted so that a given node in the vector is conditionally independent of subsequent nodes, given 
the previous ones. 


Display 


show (d::AbstractDependent) 
Write a text representation of nodal values and attributes to the current output stream. 
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showall (d::AbstractDependent) 
Write a verbose text representation of nodal values and attributes to the current output stream. 


Initialization 


setmonitor! (d::AbstractDependent, monitor: :Bool) 
setmonitor! (d::AbstractDependent, monitor: : Vector[Int] ) 
Specify node elements to be included in monitored MCMC sampler output. 


Arguments 
*d : node whose elements contain sampled MCMC values. 
«monitor : boolean indicating whether all elements are monitored, or vector of element-wise indices of 
elements to monitor. 
Value 


Returns d with its nonitor field updated to reflect the specified monitoring. 


Node Operations 


logpaf (d::AbstractDependent, transform: :Bool=false ) 
logpaf (d::AbstractDependent, x, transform: :Bool=false ) 
Evaluate the log-density function for a node. In this method, no density function is assumed for the node, and a 


constant value of 0 is returned. This method function may be redefined for subtypes of Abst ract Dependent 
that have distributional specifications. 


Arguments 
*d : node for which to evaluate the log-density. 


*x : value, of the same type and shape as the node value, at which to perform the evaluation. If not specified, 
the node value is used. 


*transform: whether the evaluation is on the link-transformed scale. 


Value 
The resulting numeric value of the log-density. 


unlist (d::AbstractDependent, transform: :Bool-false) 

unlist (d::AbstractDependent, x:: Real, transform: :Bool-false) 

unlist (d::AbstractDependent, x::AbstractArray, transform: :Bool-false) 

relist (d::AbstractDependent, x::AbstractArray, transform: :Bool=false ) 
Extract (unlist) node values to a vector, or re-assemble (relist) values to be put into a node. In this generic 
method, all values are listed. The methods are used internally for the extraction of unique stochastic node values 
to sample, and can be redefined to implement different behaviors for Abst ract Dependent subtypes. 


Arguments 
ed : node for which to unlist or relist values. 
ex : values to be listed. If not specified, the node values are used. 


*t ransform: whether to apply a link or inverse-link transformation to the values. In this generic method, 
transformations are defined to be the identity function. 


Value 


Returns unmodified x values as a vector (unlist) or in the same shape as the specified node (relist). 
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2.3.3 Logical 


The Logical types inherit fields and method functions from the Abst ractDependent type, and adds the con- 
structors and methods listed below. It is designed for nodes that are deterministic functions of model parameters and 


data. 


Declarations 


type ScalarLogical <: ScalarVariate 
type ArrayLogical(N) «: ArrayVariate{N} 
typealias AbstractLogical Union{ScalarLogical, ArrayLogical] 


Fields 


value: values of type Float 64 for ScalarLogical nodes and Array {Float64} forArrayLogical 
nodes that represent samples from a target distribution. 


symbol: : Symbol : identifying symbol for the node. 


monitor: :Vector{Int} : indices identifying elements of the value field to include in monitored MCMC 
sampler output. 


eval::Function: function for updating values stored in value. 


sources: :Vector {Symbol} : other nodes upon whom the values of this one depends. 


targets::Vector{Symbol} : Dependent nodes that depend on this one. Elements of targets are 
topologically sorted so that a given node in the vector is conditionally independent of subsequent nodes, given 
the previous ones. 


Constructors 


Logical (f::Function, monitor::Union{ Bool, Vector{Int}}=true) 
Logical (d::Integer, f:: Function, monitor: : Union[ Bool, Vector{Int}}=true) 


Construct a Logical object that defines a logical model node. 
Arguments 
*d : number of dimensions for array nodes. 


ef : function whose untyped arguments are the other model nodes upon which this one depends. The 
function may contain any valid julia expression or code block. It will be saved in the eval field of the 
constructed logical node and should return a value in the same type as and with which to update the node’s 
value field. 


«monitor : boolean indicating whether all elements are monitored, or vector of element-wise indices of 
elements to monitor. 


Value 


Returns an ArrayLogical if the dimension argument d is specified, and à ScalarLogical if 
not. 


Example 


See the Model Specification section of the tutorial. 
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Initialization 
setinits! (/::AbstractLogical, m::Model, ::Any=nothing) 
Set initial values for a logical node. 
Arguments 
*1 : logical node to which to assign initial values. 
*m : model containing the node. 
Value 


Returns the result of a call to update! (1, m). 


Node Operations 
update! (/::AbstractLogical, m::Model) 
Update the values of a logical node according to its relationship with others in a model. 
Arguments 
e1 : logical node to update. 
*m : model containing the node. 
Value 


Returns the node with its values updated. 


2.3.4 Stochastic 


The Stochastic types inherit fields and method functions from the Abst ractDependent type, and adds the 
additional ones listed below. It is designed for model parameters or data that have distributional or likelihood specifi- 
cations, respectively. Its stochastic relationship to other nodes and data structures is represented by the structure stored 
in distr field. 


Declarations 


type ScalarStochastic «: ScalarVariate 
type ArrayStochastic{N} <: ArrayVariate{N} 
typealias AbstractStochastic Union{ScalarStochastic, ArrayStochastic} 


Fields 


e value : values of type Float64 for ScalarStochastic nodes and Array{Float64} for 
ArrayStochastic nodes that represent samples from a target distribution. 


symbol: : Symbol : identifying symbol for the node. 


monitor::Vector(Int]: indices identifying elements of the value field to include in monitored MCMC 
sampler output. 


e eval::Function: function for updating the distr field for the node. 


sources: :Vector(Symbol) : other nodes upon whom the distributional specification for this one de- 
pends. 
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e targets::Vector(Symbol): Dependent nodes that depend on this one. Elements of targets are 
topologically sorted so that a given node in the vector is conditionally independent of subsequent nodes, given 
the previous ones. 


e distr: distributional specification of type UnivariateDistribution for ScalarStochastic nodes 
and DistributionStruct for ArrayStochastic nodes. 


Distribution Structures 


The DistributionStruct alias defines the types of distribution structures supported for 
AbstractStochastic nodes. Single Distribution types from the Distributions section, arrays of 
UnivariateDistribution, and arrays of MultivariateDistribution objects are supported. When a 
MultivariateDistribution array is specified for a stochastic node, the node is assumed to be one dimension 
bigger than the array, with the last dimension containing values from the distributions stored in the previous 
dimensions. Such arrays may contain distributions of different lengths. Model specification syntax for all three types 
of distribution structures can be seen in the Birats Example. 


typealias DistributionStruct Union{Distribution, 
Array{UnivariateDistribution}, 
Array{MultivariateDistribution} } 


Constructors 


Stochastic (f:: Function, monitor::Union{Bool, Vector{Int}}=true ) 
Stochastic (d:: Integer, f:: Function, monitor::Union{Bool, Vector{Int}}=true ) 
Construct a Stochastic object that defines a stochastic model node. 


Arguments 
*d : number of dimensions for array nodes. 


ef : function whose untyped arguments are the other model nodes upon which this one depends. The 
function may contain any valid julia expression or code block. It will be saved in the eval field of the 
constructed stochastic node and should return a Dist ributionStruct object to be stored in the node’s 
distr field. 


«monitor : boolean indicating whether all elements are monitored, or vector of element-wise indices of 
elements to monitor. 


Value 


Returns an ArrayStochastic if the dimension argument d is specified, and a 
ScalarStochastic if not. 


Example 


See the Model Specification section of the tutorial. 


Initialization 
setinits! (s::Stochastic, m::Model, x=nothing ) 
Set initial values for a stochastic node. 
Arguments 
es : stochastic node to which to assign initial values. 


*m : model containing the node. 
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*x : values to assign to the node. 
Value 


Returns the node with its assigned initial values. 


Node Operations 


logpdf (s::AbstractStochastic, transform::Bool=false) 
logpdf (s::AbstractStochastic, x, transform: :Bool=false ) 
Evaluate the log-density function for a stochastic node. 


Arguments 
es : stochastic node for which to evaluate the log-density. 


ex : value, of the same type and shape as the node value, at which to perform the evaluation. If not specified, 
the node value is used. 


*transform: whether the evaluation is on the link-transformed scale. 
Value 
The resulting numeric value of the log-density. 


rand (s::AbstractStochastic ) 
Draw a sample from the distributional specification on a stochastic node. 


Arguments 
*s : stochastic node from which to generate a random sample. 
Value 
Returns the sampled value(s). 


unlist (s::AbstractStochastic, transform::Bool=false) 

unlist (s::AbstractStochastic, x::Real, transform::Bool=false) 

unlist (s::AbstractStochastic, x::AbstractArray, transform: :Bool=false) 

relist (s::AbstractStochastic, x::AbstractArray, transform: :Bool=false) 
Extract (unlist) stochastic node values to a vector, or re-assemble (relist) values into a format that can be put 
into a node. These methods are used internally to extract the unique and sampled values of stochastic nodes. 
They are used, for instance, to extract only the unique, upper-triangular portions of (symmetric) covariance 
matrices and only the sampled values of Array {MultivariateDistribution} specifications whose 
distributions may be of different lengths. 


Arguments 
es : stochastic node for which to unlist or relist values. 
ex : values to be listed. If not specified, the node values are used. 


*transform: whether to apply a link transformation, or its inverse, to map values in a constrained 
distributional support to an unconstrained space. Supports for continuous, univariate distributions and 
positive-definite matrix distributions (Wishart or inverse-Wishart) are transformed as follows: 


—Lower and upper bounded: scaled and shifted to the unit interval and logit-transformed. 
—Lower bounded: shifted to zero and log-transformed. 
—Upper bounded: scaled by -1, shifted to zero, and log-transformed. 


—Positive-definite matrix: compute the (upper-triangular) Cholesky decomposition, and return it with 
the diagonal elements log-transformed. 
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Value 


Returns the extracted x values as a vector or the re-assembled values in the same shape as the speci- 
fied node. 


update! (s::AbstractStochastic, m::Model) 
Update the values of a stochastic node according to its relationship with others in a model. 


Arguments 
es : stochastic node to update. 
em : model containing the node. 
Value 


Returns the node with its values updated. 


2.3.5 Distributions 


Given in this section are distributions, as provided by the Distributions [1] and Mamba packages, supported for the 


specification of Stochastic nodes. Truncated versions of continuous univariate distributions are also supported. 


Univariate Distributions 


Distributions Package Univariate Types 


The following univariate types from the Distributions package are supported. 


Arcsine DiscreteUniform InverseGamma NoncentralChisq SymTriangularDist 
Bernoulli Edgeworth InverseGaussian NoncentralF TDist 

Beta Epanechnikov Kolmogorov NoncentralHypergeometric TriangularDist 
BetaPrime Erlang KSDist NoncentralT Triweight 
Binomial Exponential KSOneSided Normal Uniform 
Biweight FDist Laplace NormalCanon VonMises 
Categorical Frechet Levy Pareto Weibull 

Cauchy Gamma Logistic PoissonBinomial 

Chi Geometric LogNormal Poisson 

Chisq Gumbel NegativeBinomial Rayleigh 

Cosine Hypergeometric NoncentralBeta Skellam 


Flat Distribution 


A Flat distribution is supplied with the degenerate probability density function: 


f(z)ec1, —0o0<x<oo. 


Flat() # Flat distribution 


User-Defined Univariate Distributions 


New known, unknown, or unnormalized univariate distributions can be created and added 
Mamba as subtypes of the Distributions package ContinuousUnivariateDistribution 


to 
or 
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DiscreteUnivariateDistribution types. Mamba requires only a partial implementation of the method 
functions described in the full instructions for creating univariate distributions. The specific workflow is given below. 


1. Create a quote block for the new distribution. Assign the block a variable name, say extensions, preceded 
by the @everywhere macro to ensure compatibility when julia is run in multi-processor mode. 


2. The Distributions package contains types and method definitions for new distributions. Load the package and 
import the package's methods (indicated below) to be extended. 


3. Declare the new distribution subtype, say D, within the block. Any constructors explicitly defined for the subtype 
should accept un-typed or abstract-type (Real, AbstractArray, or DenseArray) arguments. Implement- 
ing constructors in this way ensures that they will be callable with the Mamba Stochastic and Logical 


types. 
4. Extend/define the following Distributions package methods for the new distribution D. 


minimum (d::D) 
Return the lower bound of the support of d. 


maximum (d::D) 
Return the upper bound of the support of d. 


logpdf (d::D, x::Real) 
Return the normalized or unnomalized log-density evaluated at x. 


5. Test the subtype. 


6. Add the quote block (new distribution) to Mamba with the following calls. 


using Mamba 


@everywher val (extensions) 


Below is a univariate example based on the linear regression model in the Tutorial. 


## Define a new univariate Distribution type for Mamba. 
## The definition must be placed within an unevaluated quote block. 
@everywher xtensions = quote 


## Load needed packages and import methods to be extended 
using Distributions 
import Distributions: minimum, maximum, logpdf 


## Type declaration 

type NewUnivarDist <: ContinuousUnivariateDistribution 
mu: :Float64 
sigma: :Float64 

end 


## The following method functions must be implemented 

## Minimum and maximum support values 

minimum(d::NewUnivarDist) = -Inf 

maximum(d::NewUnivarDist) = Inf 

## Normalized or unnormalized log-density value 

function logpdf(d::NewUnivarDist, x::Real) 
-log(d.sigma) - 0.5 * ((x - d.mu) / d.sigma)^2 

end 


end 


## Test the extensions in a temporary module (optional) 
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module Testing end 

eval(Testing, extensions) 

d = Testing.NewUnivarDist(0.0, 1.0) 
Testing.minimum (d) 
Testing.maximum (d) 
Testing.insupport (d, 2.0) 
Testing.logpdf (d, 2.0) 


## Add the extensions 
using Mamba 
@everywher val (extensions) 


## Implement a Mamba model using the new distribution 
model - Model( 


y = Stochastic(1l, 
(mu, s2) -> 
begin 
sigma = sqrt (s2) 
UnivariateDistribution[ 
NewUnivarDist (mu[i], sigma) for i in 1:length (mu) 


mu = Logical(1l, 
(xmat, beta) -> 
xmat * beta, 
false 
) YA 


beta = Stochastic(l, 
() -> MvNormal(2, sqrt(1000)) 


S2 = Stochastic( 
() -> InverseGamma(0.001, 0.001) 


## Sampling Scheme 
scheme = [NUTS (:beta), 
Slice(:s2, 3.0)] 


## Sampling Scheme Assignment 
setsamplers! (model, scheme) 


## Data 
line = Dict{Symbol, Any} ( 
ix => [1, 2, 3, 4, 5], 
¿y => [1, 3, 3, 3, 5] 
) 
line[:xmat] = [ones(5) line[:x]] 


## Initial Values 
inits = [ 
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Dict{Symbol, Any} ( 
¿y => line[:yl, 
:beta => rand(Normal (0, 1), 2), 
:s2 => rand(Gamma(1, 1)) 

) 


for i in 1:3 


## MCMC Simulation 
sim = mcmc(model, line, inits, 10000, burnin-250, thin=2, chains=3) 
describe (sim) 


Multivariate Distributions 


Distributions Package Multivariate Types 


The following multivariate types from the Distributions package are supported. 


Dirichlet MvNormal MvTDist Multinomial MvNormalCanon VonMisesFisher 


Block-Diagonal Multivariate Normal Distribution 


A Block-Diagonal Multivariate Normal distribution is supplied with the probability density function: 


1 1 _ 
fein. E) = caiga t ( TEMENE 2 , —o «2 < 00, 
where 
x1 0 0 
$ 0 >) 0 
0 0 Bn 


BDiagNormal (mu, C) # multivariate normal with mean vector mu and block- 
# diagonal covariance matrix Sigma such that 
# length (mu) = dim(Sigma), and Sigma 1 = ... = Sigma m = C 
# for a matrix C or Sigma 1 = C[1], ..., Sigma m = C[m] 
# 


for a vector of matrices C. 


User-Defined Multivariate Distributions 


New known, unknown, or unnormalized multivariate distributions can be created and added to 
Mamba as subtypes of the Distributions package ContinuousMultivariateDistribution or 
DiscreteMultivariateDistribution types. Mamba requires only a partial implementation of the 
method functions described in the full instructions for creating multivariate distributions. The specific workflow is 
given below. 


1. Create a quote block for the new distribution. Assign the block a variable name, say extensions, preceded 
by the @everywhere macro to ensure compatibility when julia is run in multi-processor mode. 


2. The Distributions package contains types and method definitions for new distributions. Load the package and 
import the package’s methods (indicated below) to be extended. 
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3. Declare the new distribution subtype, say D, within the block. Any constructors explicitly defined for the subtype 
should accept un-typed or abstract-type (Real, AbstractArray, or DenseArray) arguments. Implement- 
ing constructors in this way ensures that they will be callable with the Mamba Stochastic and Logical 


types. 
4. Extend/define the following Distributions package methods for the new distribution D. 


length (d::D) 
Return the sample space size (dimension) of d. 


insupport (T«:Real) (d::D, x::AbstractVector{T}) 
Return a logical indicating whether x is in the support of d. 


_logpdf(T<:Real) (d::D, x::AbstractVector{T}) 
Return the normalized or unnomalized log-density evaluated at x. 


5. Test the subtype. 


6. Add the quote block (new distribution) to Mamba with the following calls. 


using Mamba 
@everywhere eval (extensions) 


Below is a multivariate example based on the linear regression model in the Tutorial. 


## Define a new multivariate Distribution type for Mamba. 
## The definition must be placed within an unevaluated quote block. 
@everywher xtensions = quote 


## Load needed packages and import methods to be extended 
using Distributions 
import Distributions: length, insupport, _logpdf 


## Type declaration 

type NewMultivarDist <: ContinuousMultivariateDistribution 
mu: :VectoríFloat64) 
sigma: :Float64 

end 


## The following method functions must be implemented 


## Dimension of the distribution 
length(d::NewMultivarDist) = length(d.mu) 


## Logical indicating whether x is in the support 

function insupport{T<:Real} (d: :NewMultivarDist, x::AbstractVector{T}) 
length(d) == length(x) && all(isfinite(x) ) 

end 


## Normalized or unnormalized log-density value 

function _logpdf{T<:Real} (d: :NewMultivarDist, x::AbstractVector{T}) 
-length(x) * log(d.sigma) - 0.5 * sumabs2(x - d.mu) / d.sigma”2 

end 


end 


## Test the extensions in a temporary module (optional) 
module Testing end 

eval(Testing, extensions) 

d = Testing.NewMultivarDist([0.0, 0.0], 1.0) 
Testing.insupport(d, [2.0, 3.0]) 
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Testing.logpdf(d, [2.0, 3.0]) 


## Add the extensions 
using Mamba 
@everywher val (extensions) 


## Implement a Mamba model using the new distribution 
model - Model( 


y = Stochastic(1, 
(mu, s2) -» NewMultivarDist (mu, sqrt(s2)), 
false 

) YA 


mu = Logical(1, 
(xmat, beta) -> xmat + beta, 
false 

) , 


beta = Stochastic(l, 
() -> MvNormal (2, sqrt(1000)) 


S2 = Stochastic( 
() -> InverseGamma(0.001, 0.001) 


## Sampling Scheme 
scheme = [NUTS (:beta), 
Slice(:s2, 3.0)] 


## Sampling Scheme Assignment 
setsamplers! (model, scheme) 


## Data 

line Dict{Symbol, Any} ( 
=> [4,4 2, 3, 4, 5], 
“y => [1,9 3, 3, 3, 5] 


line[:xmat] = [ones(5) line[:x]] 


## Initial Values 
inits = [ 
Dict{Symbol, Any} ( 
¿y => line[:yl, 
:beta => rand(Normal (0, 1), 2), 
:s2 => rand(Gamma (1, 1)) 
) 
for i in 1:3 


## MCMC Simulation 
sim = mcmc(model, line, inits, 10000, burnin-250, thin=2, chains=3) 
describe (sim) 
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Matrix-Variate Distributions 


Distributions Package Matrix-Variate Types 


The following matrix-variate types from the Distributions package are supported. 


InverseWishart Wishart 


2.3.6 Sampler 


The Sampler type stores model-based Sampling Functions for use in the Mamba Gibbs sampling scheme. Developers 
can use it as a wrapper for calling stand-alone samplers or as a structure for implementing self-contained samplers. 


Declaration 


type Sampler{T} 


Fields 


e params: :Vector(Symbol) : stochastic nodes in the block being updated by the sampler. 
e eval: :Function: sampling function that updates values of the params nodes. 
e tune: :T : tuning parameters needed by the sampling function. 


e targets: :Vector(Symbol) : Dependent nodes that depend on and whose states must be updated after 
params. Elements of targets are topologically sorted so that a given node in the vector is conditionally 
independent of subsequent nodes, given the previous ones. 


Constructors 


Sampler (param::Symbol, f:: Function, tune::Any=Dict()) 
Sampler (params::Vector[Symbol], f:: Function, tune::Any=Dict()) 
Construct a Sampler object that defines a sampling function for a block of stochastic nodes. 


Arguments 
*param/params : node(s) being block-updated by the sampler. 


ef : function for the eval field of the constructed sampler and whose arguments are the other model nodes 
upon which the sampler depends, typed argument model: :Mode1 that contains all model nodes, and/or 
typed argument block: : Integer that is an index identifying the corresponding sampling function in a 
vector of all samplers for the associated model. Through the arguments, all model nodes and fields can be 
accessed in the body of the function. The function may return an updated sample for the nodes identified in 
its params field. Such a return value can be a structure of the same type as the node if the block consists 
of only one node, or a dictionary of node structures with keys equal to the block node symbols if one or 
more. Alternatively, a value of nothing may be returned. Return values that are not nothing will 
be used to automatically update the node values and propagate them to dependent nodes. No automatic 
updating will be done if nothing is returned. 


*tune : tuning parameters needed by the sampling function. 
Value 


Returns a Sampler {typeof (tune) } type object. 
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Example 


See the Sampling Schemes section of the tutorial. 


Display 


show (s::Sampler) 
Write a text representation of the defined sampling function to the current output stream. 


showall (s::Sampler) 
Write a verbose text representation of the defined sampling function to the current output stream. 


2.3.7 SamplerVariate 


The SamplerVariate type is designed to store simulated values from and tuning parameters for stand-alone Sam- 
pling Functions. It is a parametric type whose parameter can be any subtype of the abstract Samp lerTune type and 
serves to identify the family of sampling functions to which the variate belongs. 


Declaration 


abstract SamplerTune 
type SamplerVariate{T<:SamplerTune} «: VectorVariate 


Fields 


e value::VectoríFloat64] : simulated values. 


e tune: : T : tuning parameters. Type T is assumed to have a constructor T (value: :Vector{Float64}) 
that can be called with the value field of the variate to instantiate the parameters. 


Constructors 


SamplerVariate (x-:AbstractVector{ U<:Real}, tune::SamplerTune) 
SamplerVariate{T<:SamplerTune} (x::AbstractVector[U<:Real], tune::T) 
SamplerVariate{T<:SamplerTune} (x::AbstractVector[U<:Real)) 

Construct a SamplerVariate object for storing simulated values and tuning parameters. 


Arguments 
*x : simulated values. 


*tune : tuning parameters. If not specified, the tuning parameter constructor is called with the value 
field of the variate to instantiate the parameters. 


*T : explicit tuning parameter type for the variate. If not specified, the type is inferred from the tune 
argument. 


Value 


Returns a SamplerVariate{T} type object with fields containing the values supplied to arguments x and 
tune. 


SamplerVariate (m::Model, block:: Integer, transform: :Bool=false ) 
Construct a SamplerVariate object for a model-based sampler. 


Arguments 
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em : model containing nodes to be sampled. 


“block : index to a sampling block of type Sampler(T«:SamplerTune] that contains simulated 
values and tuning parameters with which to construct the variate. 


*transform: whether to apply a link transformation to the simulated values in the construction. 
Value 


Returns a SamplerVariate{T} type object with fields containing the node values and tuning parameters 
from the specified sampling block. 


2.3.8 Model 


The Model type is designed to store the set of all model nodes, including parameter set O as denoted in the Mamba 
Gibbs sampling scheme. In particular, it stores Dependent type objects in its nodes dictionary field. Valid models 
are ones whose nodes form directed acyclic graphs (DAGs). Sampling functions ( f; a are saved as Sampler 
objects in the vector of field samplers. Vector elements j = 1,..., B correspond to sampling blocks (O; bun 


Declaration 


type Model 


Fields 


nodes::Dictí(Symbol, Any}: all input, logical, and stochastic model nodes. 
samplers::Vector{Sampler} : sampling functions for updating blocks of stochastic nodes. 


states: :Vector{ModelState} : states of chains at the end of an MCMC run in a possible series of runs, 
where ModelState has fields value: : Vector{Float 64} and tune: : Vector {Any } to store the last 
values of sampled nodes and block-sampler tuning parameters, respectively. 


iter: :Int : current MCMC draw from the target distribution. 
burnin: :Int : number of initial draws to discard as a burn-in sequence to allow for convergence. 
hasinputs: :Bool: whether values have been assigned to input nodes. 


hasinits::Bool: whether initial values have been assigned to stochastic nodes. 


Constructor 


Model (; iter::Integer=0, burnin::Integer=0, samplers:: Vector[Sampler]-Sampler[], nodes...) 


Construct a Mode1 object that defines a model for MCMC simulation. 

Arguments 
*iter : current iteration of the MCMC simulation. 
*burnin: number of initial draws to be discarded as a burn-in sequence to allow for convergence. 
*samplers : block-specific sampling functions. 


*nodes... : arbitrary number of user-specified arguments defining logical and stochastic nodes in the 
model. Argument values must be Logical or Stochastic type objects. Their names in the model will 
be taken from the argument names. 


Value 
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Returns a Model type object. 
Example 


See the Model Specification section of the tutorial. 


MCMC Engine 


meme (m:: Model, inputs::Dict{Symbol}, inits:: Vector[Dict[Symbol, Any}}, iters::Integer; burnin::Integer=0, 
thin::Integer- 1, chains: :Integer- 1, verbose::Bool-true) 
meme (mc: :ModelChains, iters:: Integer; verbose::Bool-true) 
Simulate MCMC draws for a specified model. 


Arguments 
*m : specified model. 
emc : chains from a previous call to mcmc for which to simulate additional draws. 
*inputs : values for input model nodes. Dictionary keys and values should be given for each input node. 


*inits : dictionaries that contain initial values for stochastic model nodes. Dictionary keys and values 
should be given for each stochastic node. Consecutive runs of the simulator will iterate through the vector's 
dictionary elements. 


*iters : number of draws to generate for each simulation run. 
*burnin: numer of initial draws to discard as a burn-in sequence to allow for convergence. 
*thin : step-size between draws to output. 
*chains : number of simulation runs to perform. 
*verbose : whether to print sampler progress at the console. 
Value 
A ModelChains type object of simulated draws. 
Example 


See the MCMC Simulation section of the tutorial. 


Indexing 


getindex (m::Model, nodekey::Symbol) 
Returns a model node identified by its symbol. The syntax m[nodekey] is converted to get index (m, 
nodekey). 


Arguments 
*m : model containing the node to get. 
*nodekey : node to get. 
Value 
The specified node. 


keys (m:: Model) 
keys (m::Model, ntype::Symbol, at...) 
Extract the symbols (keys) for all existing nodes or for nodes of a specified type. 


Arguments 
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*m : model containing the nodes of interest. 
*ntype [type of nodes to return. Options are] 


— :all: all input, logical, and stochastic model nodes. 


:assigned: nodes that have been assigned values. 


— :block : stochastic nodes being updated by the sampling block(s) at : : Integer=0 (default: 
all blocks). 


— :dependent : logical and stochastic (dependent) nodes in topologically sorted order. 
— :independent or : input : input (independent) nodes. 

— :logical: logical nodes. 

— :monitor: stochastic nodes being monitored in MCMC sampler output. 

— :output : stochastic nodes upon which no other stochastic nodes depend. 


= :source : nodes upon which the node at::Symbol or vector of nodes 
at::Vector {Symbol} depends. 


— :stochastic: stochastic nodes. 


— :target : topologically sorted nodes that depend on the sampling block(s) at : : Integer=0 
(default: all blocks), node at : : Symbol, or vector of nodes at : : Vector {Symbol}. 


*at... : additional positional arguments to be passed to the nt ype options, as described above. 
Value 


A vector of node symbols. 


Display 
draw (m::Model; filename::AbstractString=” ") 
Draw a Graph Viz DOT-formatted graph representation of model nodes and their relationships. 
Arguments 
*m : model for which to construct a graph. 


*filename : external file to which to save the resulting graph, or an empty string to draw to standard 
output (default). If a supplied external file name does not include a dot (.), the file extension . dot will 
be appended automatically. 


Value 


The model drawn to an external file or standard output. Stochastic, logical, and input nodes will 
be represented by ellipses, diamonds, and rectangles, respectively. Nodes that are unmonitored in 
MCNC simulations will be gray-colored. 


Example 
See the Directed Acyclic Graphs section of the tutorial. 


graph (m:: Model) 
Construct a graph representation of model nodes and their relationships. 


Arguments 
*m : model for which to construct a graph. 


Value 
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Returns a GenericGraph type object as defined in the Graphs package. 


graph2dot (m::Model) 
Draw a GraphViz DOT-formatted graph representation of model nodes and their relationships. 


Arguments 
*m : model for which to construct a graph. 
Value 


A character string representation of the graph suitable for in-line processing. Stochastic, logical, and 
input nodes will be represented by ellipses, diamonds, and rectangles, respectively. Nodes that are 
unmonitored in MCMC simulations will be gray-colored. 


Example 
See the Directed Acyclic Graphs section of the tutorial. 


show (m::Model) 
Write a text representation of the model, nodes, and attributes to the current output stream. 


showall (m:: Model) 
Write a verbose text representation of the model, nodes, and attributes to the current output stream. 


Initialization 
setinits! (m::Model, inits::Dict{Symbol, Any}) 
Set the initial values of stochastic model nodes. 
Arguments 
em: model with nodes to be initialized. 


*inits : initial values for stochastic model nodes. Dictionary keys and values should be given for each 
stochastic node. 


Value 

Returns the model with stochastic nodes initialized and the iter field set equal to 0. 
Example 

See the Development and Testing section of the tutorial. 


setinputs! (m::Model, inputs::Dict{Symbol, Any]) 
Set the values of input model nodes. 


Arguments 
*m : model with input nodes to be assigned. 
*inputs : values for input model nodes. Dictionary keys and values should be given for each input node. 
Value 
Returns the model with values assigned to input nodes. 
Example 
See the Development and Testing section of the tutorial. 


setsamplers! (m::Model, samplers::Vector(T«:Sampler]) 
Set the block-samplers for stochastic model nodes. 


Arguments 
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em : model with stochastic nodes to be sampled. 
*samplers : block-specific samplers. 
Values: 
Returns the model updated with the block-samplers. 
Example 


See the Model Specification and MCMC Simulation sections of the tutorial. 


Parameter Block Operations 
gettune (m::Model, block::Integer=0) 
Get block-sampler tuning parameters. 
Arguments 
em : model with block-samplers. 
“block : block for which to get the tuning parameters (default: all blocks). 
Value 


A Vector {Any} of all block-specific tuning parameters if b1ock=0, and turning parameters for 
the specified block otherwise. 


gradlogpdf (m::Model, block::Integer=0, transform: :Bool=false; dtype::Symbol=:forward ) 
gradlogpdf (m::Model, x::AbstractArray{T<:Real}, block: :Integer=0, transform: :Bool=false; 
dtype::Symbol=:forward) 
gradlogpdf! (m::Model, x::AbstractArray{T<:Real}, block: :Integer=0, transform: :Bool=false; 
dtype: :Symbol=:forward ) 
Compute the gradient of log-densities for stochastic nodes. 


Arguments 
em : model containing the stochastic nodes for which to compute the gradient. 


“block : sampling block of stochastic nodes for which to compute the gradient (default: all stochastic 
nodes). 


*x : value (possibly different than the current one) at which to compute the gradient. 
*t ransform: whether to compute the gradient of block parameters on the link-transformed scale. 
*dtype [type of differentiation for gradient calculations. Options are] 
— :central: central differencing. 
— : forward: forward differencing. 
Value 


The resulting gradient vector. Method gradlogpdf! () additionally updates model m with sup- 
plied values x. 


Note 


Numerical approximation of derivatives by central and forward differencing is performed with the 
Calculus package [93]. 


logpdf (m::Model, block: :Integer=0, transform: :Bool=false) 
logpdf (m::Model, nodekeys::Vector{Symbol}, transform: :Bool=false) 
logpdf (m::Model, x::AbstractArray[T<:Real), block::Integer=0, transform: :Bool=false) 
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logpdf! (m::Model, x::AbstractArray{T<:Real}, block::Integer=0, transform: :Bool=false) 
Compute the sum of log-densities for stochastic nodes. 


Arguments 

em : model containing the stochastic nodes for which to evaluate log-densities. 

“block : sampling block of stochastic nodes over which to sum densities (default: all stochastic nodes). 
*nodekeys : nodes over which to sum densities. 


ex : value (possibly different than the current one) at which to evaluate densities. 


*transform: whether to evaluate evaluate log-densities of block parameters on the link-transformed 
scale. 


Value 


The resulting numeric value of summed log-densities. Method logpdf! () additionally updates 
model m with supplied values x. 


simulate! (m::Model, block::Integer=0) 
Simulate one MCMC draw from a specified model. 


Argument: 

em : model specification. 

“block : block for which to simulate an MCMC draw (default: all blocks). 
Value 


Returns the model updated with the MCMC draw and, in the case of block=0, the iter field 
incremented by 1. 


Example 
See the Development and Testing section of the tutorial. 


unlist (m::Model, block::Integer=0, transform: :Bool=false) 
unlist (m::Model, nodekeys::Vector{Symbol}, transform: :Bool=false) 
relist (m::Model, x::AbstractArray{T<:Real}, block: :Integer=0, transform: :Bool=false) 
relist (m::Model, x::AbstractArray{T<:Real}, nodekeys::Vector{Symbol}, transform: :Bool=false ) 
relist! (m::Model, x::AbstractArray{T<:Real}, block::Integer=0, transform: :Bool=false ) 
relist! (m::Model, x::AbstractArray{T<:Real}, nodekey::Symbol, transform::Bool=false ) 
Convert (unlist) sets of logical and/or stochastic node values to vectors, or reverse (relist) the process. 


Arguments 

em : model containing nodes to be unlisted or relisted. 

“block : sampling block of nodes to be listed (default: all blocks). 
*nodekey /nodekeys : node(s) to be listed. 


ex : values to re-list. 


*t ransform: whether to apply a link transformation in the conversion. 
Value 


The unlist methods return vectors of concatenated node values, relist return dictionaries of 
symbol keys and values for the specified nodes, and relist! return their model argument with 
values copied to the nodes. 


update! (m:: Model, block::Integer=0) 
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update! (m::Model, nodekeys:: Vector[ Symbol] ) 
Update values of logical and stochastic model node according to their relationship with others in a model. 


Arguments 
*m : mode with nodes to be updated. 
“block : sampling block of nodes to be updated (default: all blocks). 
enodekeys : nodes to be updated in the given order. 

Value 


Returns the model with updated nodes. 


2.3.9 Chains 


AbstractChains subtypes store output from one or more runs (chains) of an MCMC sampler. They serve as con- 
tainers for output generated by the mcmc () function, and supply methods for convergence diagnostics and posterior 
inference. Moreover, they can be used as stand-alone containers for any user-generated MCMC output, and are thus a 
julia analogue to the boa [82/[83] and coda [69][70] R packages. 


Declarations 


abstract AbstractChains 
immutable Chains «: AbstractChains 
immutable ModelChains «: AbstractChains 


Fields 


e value::Array(Float64, 3) : 3-dimensional array of sampled values whose first, second, and third 
dimensions index the iterations, parameter elements, and runs of an MCMC sampler, respectively. 


e range: :Range(Int ) : range of iterations stored in the rows of the value array. 
* names::Vector{AbstractString} : names assigned to the parameter elements. 
e chains: :Vector(Int) : indices to the MCMC runs. 


e model: :Model : model from which the sampled values were generated (ModelChains only). 


Constructors 


Chains (iters::Integer,  params::Integer; start: :Integer- 1, thin::Integer=1, | chains::Integer- 1, 
names:: Vector{T<:AbstractString ]-AbstractString[]) 
Chains (value::Array(T«:Real, 3}; start::Integer=1, thin::Integer=1, names:: Vector[ U«:AbstractString] 2AbstractString[ ], 


chains:: Vector( V« Integer] -Int[]) 
Chains (value::Matrix{T<:Real}; start::Integer=1, thin::Integer=1, names: : Vector{ U«:AbstractString ] -AbstractString[ ], 
chains::Integerz 1 ) 
Chains (value::Vector(T«:Real];  start::Integer=1,  thin::Integerz1, names::AbstractString=”Paraml”, 
chains::Integerz 1 ) 
ModelChains (c::Chains, m:: Model) 
Construct a Chains or ModelChains object that stores MCMC sampler output. 


Arguments 


*iters : total number of iterations in each sampler run, of which length (start:thin:iters) 
outputted iterations will be stored in the object. 
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*params : number of parameters to store. 


*value : array whose first, second (optional), and third (optional) dimensions index outputted iterations, 
parameter elements, and runs of an MCMC sampler, respectively. 


*start : number of the first iteration to be stored. 
*thin : number of steps between consecutive iterations to be stored. 
«Chains : number of simulation runs for which to store output, or indices to the runs (default: 1, 2, ...). 
*names : names to assign to the parameter elements (default: "Param1", "Param2", ...). 
em : model for which simulated values were generated. 
Value 


Returns an object of type Chains or ModelChains according to the name of the constructor 
called. 


Example 


See the AMM, AMWG, NUTS, and Slice examples. 


Indexing and Concatenation 


cat (dim::Integer, chains::AbstractChains... ) 

veat (chains: :AbstractChains... ) 

heat (chains: :AbstractChains... ) 
Concatenate input MCMC chains along a specified dimension. For dimensions other than the specified one, all 
input chains must have the same sizes, which will also be the sizes of the output chain. The size of the output 
chain along the specified dimension will be the sum of the sizes of the input chains in that dimension. vcat 
concatenates vertically along dimension 1, and has the alternative syntax [chainl; chain2; ...]. hcat 
concatenates horizontally along dimension 2, and has the alternative syntax [chainl chain2 ...]. 


Arguments 
*dim: dimension (1, 2, or 3) along which to concatenate the input chains. 
*chains: chains to concatenate. 
Value 
A Chains object containing the concatenated input. 
Example 
See the readcoda () example. 


first (c::AbstractChains) 
step (c::AbstractChains) 
last (c::AbstractChains) 
Get the first iteration, step-size (thinning), or last iteration of MCMC sampler output. 


Arguments 
*c : sampler output for which to return results. 
Value 
Integer value of the requested iteration type. 


getindex (c::Chains, window, names, chains) 
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getindex (mc::ModelChains, window, names, chains) 
Subset MCMC sampler output. The syntax c[i, j, k] isconvertedto getindex(c, i, j, k). 


Arguments 
*c : sampler output to subset. 


ewindow : indices of the form start:stopor start:thin:stop can be used to subset iterations, 
where start and stop define a range for the subset and thin will apply additional thinning to existing 
sampler output. 


*names : indices for subsetting of parameters that can be specified as strings, integers, or booleans identi- 
fying parameters to be kept. ModelChains may additionally be indexed by model node symbols. 


*chains: indices for chains can be integers or booleans. 
A value of : can be specified for any of the dimensions to indicate no subsetting. 
Value 
Subsetted sampler output stored in the same type of object as that supplied in the call. 
Example 
See the Output Subsetting section of the tutorial. 


setindex! (c::AbstractChains, value, iters, names, chains) 
Store MCMC sampler output at a given index. The syntax c[i, j, k] = value is converted to 
setindex!(c, value, i, j, k). 


Arguments 
*c : object within which to store sampler output. 
“value: sampler output. 


*iters: iterations can be indexed as a start :stopor start:thin:stop range, a single 
numeric index, or a vector of indices; and are taken to be relative to the index range store in the 
c.range field. 


*names : indices for subsetting of parameters can be specified as strings, integers, or booleans. 
*chains: indices for chains can be integers or booleans. 
A value of : can be specified for any of the dimensions to index all corresponding elements. 
Value 
An object of the same type as c with the sampler output stored in the specified indices. 
Example 


See the AMM, AMWG, NUTS, and Slice examples. 


File /O 


read (name::AbstractString, ::Type{T<:AbstractChains} ) 
write (name: :AbstractString, c::AbstractChains) 
Read a chain from or write one to an external file. 


Arguments 


«name : file to read or write. Recommended convention is for the file name to be specified with a . jls 
extension. 


eT : chain type to read. 
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ec : chain to write. 
Value 


An AbstractChains subtype read from an external file, or a written external file containing a 
subtype. 


Example 
See the File I/O section of the tutorial. 


readcoda (output::AbstractString, index: :AbstractString ) 
Read MCMC sampler output generated in the CODA format by OpenBUGS /86/. The function only retains 
those sampler iterations at which all model parameters were monitored. 


Arguments 
*output : text file containing the iteration numbers and sampled values for the model parameters. 


*index : text file containing the names of the parameters, followed by the first and last rows in which their 
output can be found in the output file. 


Value 
A Chains object containing the read sampler output. 
Example 


The following example reads sampler output contained in the CODA files linel.out, 
linel.ind, line2.out,and line2.ind. 


using Mamba 


## Get the directory in which the example CODA files are saved 
dir dirname (@__FILE__) 


## Read the MCMC sampler output from the files 
= readcoda(joinpath(dir, "linel.out"), joinpath (dir, "linel.ind") ) 
c2 = readcoda (joinpath (dir, "line2.out"), joinpath(dir, "line2.ind") ) 


Q 
p 
| 


## Concatenate the resulting chains 
c = cat(3, cl, c2) 


## Compute summary statistics 
describe (c) 


Convergence Diagnostics 


MCMC simulation provides autocorrelated samples from a target distribution. Because of computational complexities 
in implementing MCMC algorithms, the autocorrelated nature of samples, and the need to choose initial sampling 
values at different points in target distributions; it is important to evaluate the quality of resulting output. Specifically, 
one should check that MCMC samples have converged to the target (or, more commonly, are stationary) and that the 
number of convergent samples provides sufficiently accurate and precise estimates of posterior statistics. 


Several established convergence diagnostics are supplied by Mamba. The diagnostics and their features are summa- 
rized in the table below and described in detail in the subsequent function descriptions. They differ with respect to 
the posterior statistic being assessed (mean vs. quantile), whether the application is to parameters univariately or 
multivariately, and the number of chains required for calculations. Diagnostics may assess stationarity, estimation 
accuracy and precision, or both. A more comprehensive comparative review can be found in //6/. Since diagnostics 
differ in their focus and design, it is often good practice to employ more than one to assess convergence. Note too that 
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diagnostics generally test for non-convergence and that non-significant test results do not prove convergence. Thus, 
non-significant results should be interpreted with care. 


Table 2.2: Comparative summary of features for the supplied MCMC convergence diagnostics. 


Convergence Assessments 
Diagnostic Statistic | Parameters | Chains | Stationarity | Estimation 
Gelman, Rubin, and Brooks | Mean Univariate 2+ Yes No 
Multivariate 2+ Yes No 
Geweke Mean Univariate 1 Yes No 
Heidelberger and Welch Mean Univariate 1 Yes Yes 
Raftery and Lewis Quantile | Univariate 1 Yes Yes 


Gelman, Rubin, and Brooks Diagnostics 


gelmandiag (c::AbstractChains; alpha::Real=0.05, mpsrf::Bool=false, transform: :Bool=false ) 


Compute the convergence diagnostics of Gelman, Rubin, and Brooks /31//[10] for MCMC sampler output. The 
diagnostics are designed to asses convergence of posterior means estimated with multiple autocorrelated samples 
(chains). They does so by comparing the between and within-chain variances with metrics called potential scale 
reduction factors (PSRF). Both univariate and multivariate factors are available to assess the convergence of 
parameters individually and jointly. Scale factors close to one are indicative of convergence. As a rule of 
thumb, convergence is concluded if the 0.975 quantile of an estimated factor is less than 1.2. Multiple chains are 
required for calculations. It is recommended that at least three chains be generated, each with different starting 
values chosen to be diffuse with respect to the anticipated posterior distribution. Use of multiple chains in the 
diagnostic provides for more robust assessment of convergence than is possible with single chain diagnostics. 


Arguments 


*c : sampler output on which to perform calculations. 
*alpha:quantile(1 — alpha / 2)at which to estimate the upper limits of scale reduction factors. 


empsrf: whether to compute the multivariate potential scale reduction factor. This factor will not be 
calculable if any one of the parameters in the output is a linear combination of others. 


*t ransform: whether to apply log or logit transformations, as appropriate, to parameters in the chain to 
potentially produce output that is more normally distributed, an assumption of the PSRF formulations. 


Value 


A ChainSummary type object of the form: 


immutable ChainSummary 
value::Array(Float64, 3} 
rownames::Vector{AbstractString} 
colnames::Vector{AbstractString} 
header: :AbstractString 

end 


with parameters contained in the rows of the value field, and scale reduction factors and upper-limit 
quantiles in the first and second columns. 


Example 


See the Convergence Diagnostics section of the tutorial. 
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Geweke Diagnostic 


gewekediag (c::AbstractChains; first::Real=0.1, last::Real=0.5, etype=:imse, args...) 

Compute the convergence diagnostic of Geweke /35/ for MCMC sampler output. The diagnostic is designed 
to asses convergence of posterior means estimated with autocorrelated samples. It computes a normal-based 
test statistic comparing the sample means in two windows containing proportions of the first and last itera- 
tions. Users should ensure that there is sufficient separation between the two windows to assume that their 
samples are independent. A non-significant test p-value indicates convergence. Significant p-values indicate 
non-convergence and the possible need to discard initial samples as a burn-in sequence or to simulate additional 
samples. 


Arguments 
ec : sampler output on which to perform calculations. 
“first : proportion of iterations to include in the first window. 
*last : proportion of iterations to include in the last window. 
*etype : method for computing Monte Carlo standard errors. See mcse () for options. 
*args... : additional arguments to be passed to the et ype method. 
Value 


A ChainSummary type object with parameters contained in the rows of the value field, and test 
Z-scores and p-values in the first and second columns. Results are chain-specific. 


Example 


See the Convergence Diagnostics section of the tutorial. 


Heidelberger and Welch Diagnostic 


heideldiag (c::AbstractChains; alpha::Real=0.05, eps::Real=0.1, etype=:imse, args...) 

Compute the convergence diagnostic of Heidelberger and Welch /44] for MCMC sampler output. The diagnostic 
is designed to assess convergence of posterior means estimated with autocorrelated samples and to determine 
whether a target degree of accuracy is achieved. A stationarity test is performed for convergence assessment 
by iteratively discarding 10% of the initial samples until the test p-value is non-significant and stationarity 
is concluded or until 50% have been discarded and stationarity is rejected, whichever occurs first. Then, a 
halfwidth test is performed by calculating the relative halfwidth of a posterior mean estimation interval as 
33—9/28/ (0|; where z is a standard normal quantile, 3 is the Monte Carlo standard error, and 0 is the estimated 
posterior mean. If the relative halfwidth is greater than a target ratio, the test is rejected. Rejection of the 
stationarity or halfwidth test suggests that additional samples are needed. 


Arguments 
*c : sampler output on which to perform calculations. 


*alpha : significance level for evaluations of stationarity tests and calculations of relative estimation 
interval halfwidths. 


*eps : target ratio for the relative halfwidths. 
*etype : method for computing Monte Carlo standard errors. See mcse () for options. 
*args... : additional arguments to be passed to the et ype method. 


Value 
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A ChainSummary type object with parameters contained in the rows of the value field, and 
numbers of burn-in sequences to discard, whether the stationarity tests are passed (1 — yes, 0 — 
no), their p-values (p > a implies stationarity), posterior means, halfwidths of their (1 — a)100% 
estimation intervals, and whether the halfwidth tests are passed (1 = yes, 0 = no) in the columns. 
Results are chain-specific. 


Example 


See the Convergence Diagnostics section of the tutorial. 


Raftery and Lewis Diagnostic 


rafterydiag (c::AbstractChains; q::Real=0.025, r::Real=0.005, s::Real=0.95, eps::Real=0.001) 

Compute the convergence diagnostic of Raftery and Lewis /71 ][72] for MCMC sampler output. The diagnostic 
is designed to determine the number of autocorrelated samples required to estimate a specified quantile 04, such 
that Pr(0 < 0,) = q, within a desired degree of accuracy. In particular, if Ó, is the estimand and Pr(0 < ba) = 
P, the estimated cumulative probability, then accuracy is specified in terms of r and s, where Pr(q — r « Ê, < 
q+r) = s. Thinning may be employed in the calculation of the diagnostic to satisfy its underlying assumptions. 
However, users may not want to apply the same (or any) thinning when estimating posterior summary statistics 
because doing so results in a loss of information. Accordingly, sample sizes estimated by the diagnostic tend to 
be conservative (too large). 


Arguments 
ec : sampler output on which to perform calculations. 
*q : posterior quantile of interest. 
er : margin of error for estimated cumulative probabilities. 
es : probability for the margin of error. 


*eps : tolerance within which the probabilities of transitioning from initial to retained iterations are within 
the equilibrium probabilities for the chain. This argument determines the number of samples to discard as 
a burn-in sequence and is typically left at its default value. 


Value 


A ChainSummary type object with parameters contained in the rows of the value field, and 
thinning intervals employed, numbers of samples to discard as burn-in sequences, total numbers 
(N) to burn-in and retain, numbers of independent samples that would be needed (N min), and 
dependence factors (N/Nmin) in the columns. Results are chain-specific. 


Example 


See the Convergence Diagnostics section of the tutorial. 


Posterior Summary Statistics 
autocor (c::AbstractChains; lags::Vector=[1, 5, 10, 50], relative::Bool=true) 
Compute lag-k autocorrelations for MCMC sampler output. 
Arguments 
*c : sampler output on which to perform calculations. 
*lags : lags at which to compute autocorrelations. 


“relative: whether the lags are relative to the thinning interval of the output (t rue) or relative to the 
absolute iteration numbers (false). 
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Value 


A ChainSummary type object with model parameters indexed by the first dimension of value, 
lag-autocorrelations by the second, and chains by the third. 


Example 
See the Posterior Summaries section of the tutorial. 


changerate (c::AbstractChains) 
Estimate the probability, or rate per iteration, Pr(0^ 4 0171) of a state space change for iterations i = 2,...,.N 
in MCMC sampler output. Estimation is performed for each parameter univariately as well as for the full 
parameter vector multivariately. For continuous output generated from samplers, like Metropolis-Hastings, 
whose algorithms conditionally accept candidate draws, the probability can be viewed as the acceptance rate. 


Arguments 
*c : sampler output on which to perform calculations. 
Value 


A ChainSummary type object with parameters in the rows of the value field, and the estimated 
rates in the column. Results are for all chains combined. 


Example 
See the Posterior Summaries section of the tutorial. 


cor (c::AbstractChains) 
Compute cross-correlations for MCMC sampler output. 


Arguments 
*c : sampler output on which to perform calculations. 
Value 


A ChainSummary type object with the first and second dimensions of the value field indexing 
the model parameters between which correlations. Results are for all chains combined. 


Example 
See the Posterior Summaries section of the tutorial. 


describe (c::AbstractChains; q:: Vector [0.025, 0.25, 0.5, 0.75, 0.975], etype=:bm, args...) 
Compute summary statistics for MCMC sampler output. 


Arguments 
*c : sampler output on which to perform calculations. 
eq : probabilities at which to calculate quantiles. 


*etype : method for computing Monte Carlo standard errors. See mcse () for options. 


*args... : additional arguments to be passed to the et ype method. 
Value 
Results from calls to summarystats(c, etype, args...) and quantile(c, q) are 


printed for all chains combined, and a value of nothing is returned. 
Example 


See the Posterior Summaries section of the tutorial. 
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hpd (c::AbstractChains; alpha::Real=0.05) 
Compute highest posterior density (HPD) intervals of Chen and Shao /74] for MCMC sampler output. HPD 
intervals have the desirable property of being the smallest intervals that contain a given probability. However, 
their calculation assumes unimodal marginal posterior distributions, and they are not invariant to transformations 
of parameters like central (quantile-based) posterior intervals. 


Arguments 
*c : sampler output on which to perform calculations. 
*alpha:the 100 « (1 - alpha) % interval to compute. 
Value 


A ChainSummary type object with parameters contained in the rows of the value field, and lower 
and upper intervals in the first and second columns. Results are for all chains combined. 


Example 
See the Posterior Summaries section of the tutorial. 


mcse (x::Vector( T«:Real], method::Symbol-:imse; args...) 
Compute Monte Carlo standard errors. 


Arguments 
*x : time series of values on which to perform calculations. 
«method [method used for the calculations. Options are] 


— :bm: batch means /39], with optional argument size: : Integer=100 determining the num- 
ber of sequential values to include in each batch. This method requires that the number of values 
in x is at least 2 times the batch size. 


— :imse: initial monotone sequence estimator /36/. 
— :ipse: initial positive sequence estimator /36/. 
*args... : additional arguments for the calculation method. 
Value 
The numeric standard error value. 


quantile (c::AbstractChains; q::Vector=[0.025, 0.25, 0.5, 0.75, 0.975]) 
Compute posterior quantiles for MCMC sampler output. 


Arguments 
ec : sampler output on which to perform calculations. 
eq : probabilities at which to compute quantiles. 
Value 


A ChainSummary type object with parameters contained in the rows of the value field, and 
quantiles in the columns. Results are for all chains combined. 


summarystats (c::AbstractChains; etype=:bm, args...) 
Compute posterior summary statistics for MCMC sampler output. 


Arguments 
*c : sampler output on which to perform calculations. 
*etype : method for computing Monte Carlo standard errors. See mcse () for options. 


*args... : additional arguments to be passed to the et ype method. 
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Value 


A ChainSummary type object with parameters in the rows of the value field; and the sample 
mean, standard deviation, standard error, Monte Carlo standard error, and effective sample size in the 
columns. Results are for all chains combined. 


Model-Based Inference 


dic (mc::ModelChains) 
Compute the Deviance Information Criterion (DIC) of Spiegelhalter et al. /57/ and Gelman et al. /29] from 
MCMC sampler output. 


Arguments 
*mc : sampler output from a model fit with the mcmc () function. 
Value 


A ChainSummary type object with DIC results from the methods of Spiegelhalter and Gelman in 
the first and second rows of the value field, and the DIC value and effective numbers of parameters 
in the first and second columns; where 


DIC = —2£(0) + 2p, 


such that L (©) is the log-likelihood of model outputs given the expected values of model parameters 
O, and p is the effective number of parameters. The latter is defined as pp = —2£(0) + 2£(0) for 
the method of Spiegelhalter and as py = 3 var(—2L(0)) for the method of Gelman. Results are for 
all chains combined. 


Example 
See the Posterior Summaries section of the tutorial. 


logpdf (mc::ModelChains, nodekey::Symbol) 
logpdf (mc::ModelChains, nodekeys: :Vector[Symbol]=keys(mc.model, :stochastic)) 
Compute the sum of log-densities at each iteration of MCMC output for stochastic nodes. 


Arguments 

«mc: sampler output from a model fit with the memc () function. 

enodekey/nodekeys: stochastic model node(s) over which to sum densities (default: all). 
Value 


A ModelChains object of resulting summed log-densities at each MCMC iteration of the supplied 
chain. 


predict (mc::ModelChains, nodekey::Symbol) 
predict (mc::ModelChains, nodekeys: : Vector{Symbol}=keys(mc.model, :output)) 
Generate MCMC draws from a posterior predictive distribution. 


Arguments 
emc: sampler output from a model fit with the mcmc () function. 


enodekey/nodekeys: observed Stochastic model node(s) for which to generate draws from the predic- 
tive distribution (default: all observed data nodes). 


Value 
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A ModelChains object of draws simulated at each MCMC iteration of the supplied chain. For 
observed data node y, simulation is from the posterior predictive distribution 


p(gly) = / »(jlO)p(Oly)de, 


where ij is an unknown observation on the node, p(y|O) is the data likelihood, and p(O|y) is the 
posterior distribution of unobserved parameters O. 


Example 


See the Pumps example. 


Plotting 


plot (c::AbstractChains, ptype:: Vector[ Symbol] -[:trace, :density]; legend::Bool=false, args...) 

plot (c::AbstractChains, ptype::Symbol; legend::Bool=false, args...) 
Various plots to summarize sampler output stored in Abst ract Chains subtypes. Separate plots are produced 
for each sampled parameter. 


Arguments 
*c : sampler output to plot. 
eptype [plot type(s). Options are] 


= :autocor : autocorrelation plots, with optional argument 
maxlag::Integer-round(Int, 10 * loglO(length(c.range))) determining 
the maximum autocorrelation lag to plot. Lags are plotted relative to the thinning interval of the 
output. 


— :bar: bar plots. Optional argument position::Symbol-:stack controls whether bars 
should be stacked on top of each other (default) or placed side by side ( : dodge). 


= :contour : pairwise posterior density contour plots. Optional argument 
bins::Integer-100 controls the plot resolution. 


:density : density plots. Optional argument trim: :Tuple(Real, Real)=(0.025, 
0.975) trims off lower and upper quantiles of density. 


— :mean: running mean plots. 


:mixeddensity : bar plots (: bar) for parameters with integer values within bounds defined 
by optional argument barbounds::Tuple(Real, Real}=(0, Inf), and density plots 
(:density) otherwise. 


— :trace: trace plots. 
*legend: whether to include legends in the plots to identify chain-specific results. 
*args... : additional keyword arguments to be passed to the pt ype options, as described above. 
Value 


Returns a Vector{Plot} whose elements are individual parameter plots of the specified type 
if ptype is a symbol, and a Matrix(Plot) with plot types in the rows and parameters in the 
columns if pt ype is a vector. The result can be displayed or saved to a file with draw (). 


Note 
Plots are created using the Gadfly package /48]. 


Example 
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See the Plotting section of the tutorial. 


draw (p::Array[Plot]; fmt::Symbol=:svg, filename::AbstractString=”*“, width::MeasureOrNumber=Sinch, 
height::MeasureOrNumber=8inch, nrow::Integer=3, ncol::Integer=2, byrow::Bool=true, 


ask::Bool=true) 
Draw plots produced by plot () into display grids containing a default of 3 rows and 2 columns of plots. 


Arguments 


*p : plots to be drawn. Elements of p are read in the order stored by julia (e.g. column-major 
order for matrices) and written to the display grid according to the by row argument. Grids will 
be filled sequentially until all plots have been drawn. 


*£mt [output format. Options are] 
— :pdf : Portable Document Format (.pdf). 
— :pgf : Portable Graphics Format (.pgf). 
— :png: Portable Network Graphics (.png). 
— :ps: Postscript (.ps). 
— :svg: Scalable Vector Graphics (.svg). 


*filename: file to which to save the display grids as they are drawn, or an empty string to draw 
to the display device (default). If a supplied external file name does not include a dot (.), then a 
hyphen followed by the grid sequence number and then the format extension will be appended 
automatically. In the case of multiple grids, the former file name behavior will write all grids 
to the single named file, but prompt users before advancing to the next grid and overwriting the 
file; the latter behavior will write each grid to a different file. 


ewidth/height : grid widths/heights in cm, mm, inch, pt, or px units. 
*nrow/ncol : number of rows/columns in the display grids. 
*byrow : whether the display grids should be filled by row. 


eask : whether to prompt users before displaying subsequent grids to a single named file or the 
display device. 


Value 
Grids drawn to an external file or the display device. 
Example 


See the Plotting section of the tutorial. 


2.4 Sampling Functions 


Listed below are the sampling methods for which functions are provided to simulating draws from distributions that 
can be specified up to constants of proportionalities. Model-based Sampler constructors are available for use with the 
mcmc () engine as well as stand-alone functions that can be used independently. 


2.4.1 Approximate Bayesian Computation (ABC) 


Approximate Bayesian Computation in the framework of MCMC (also known as Likelihood-Free MCMC) as pro- 
posed by /56] for simulating autocorrelated draws from a posterior distribution without evaluating its likelihood. Also 
see [81] for a thorough review of Likelihood-Free MCMC. 
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Sampler Constructor 


ABC (params::ElementOrVector[Symbol], scale:: ElementOrVector[T«: Real], summary: :Function, 


epsilon: :Real; kernel::KernelDensityType- SymUniform, dist::Function=(Tsim, Tobs) -> 
sqri(sumabs2(Tsim | -  Tobs)), — proposal::SymDistributionType=Normal, | maxdraw::Integer=1, 
nsim::Integer=1, args...) 

Construct a Sampler object for ABC sampling. Parameters are assumed to be continuous, but may be con- 
strained or unconstrained. 


Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


escale : scaling value or vector of the same length as the combined elements of nodes params for the 
proposal distribution. Values are relative to the unconstrained parameter space, where candidate draws 
are generated. 


*summary : function that takes a vector of observed or simulated data and returns a summary statistic or 
vector of statistics. 


*epsilon: target tolerance for determining how similar observed and simulated data summary statistics 
need to be in order to accept a candidate draw. Internal tolerances are adaptively tuned at each iteration to 
decrease monotonically to this target. 


*kernel : weighting kernel density of type Biweight, Cosine, Epanechnikov, Normal, 
SymTriangularDist, SymUniform, or Triweight to use in measuring similarity between ob- 
served and simulated data summary statistics. Specified epsilon determines the standard deviation of 
Normal kernels and widths of the others. 


*dist : positive function for the kernel density to compute distance between vectors of observed (Tobs) 
and simulated (Tsim) data summary statistics (default: Euclidean distance). 


*proposal : symmetric distribution of type Biweight, Cosine, Epanechnikov, Normal, 
SymTriangularDist, SymUniform, or Triweight to be centered around current parameter val- 
ues and used to generate proposal draws. Specified scale determines the standard deviations of Normal 
proposals and widths of the others. 


*maxdraw: maximum number of unaccepted candidates to draw in each call of the sampler. Draws are 
generated until one is accepted or the maximum is reached. Larger values increase acceptance rates at the 
expense of longer runtimes. 


ensim: number of data sets to simulate in deciding whether to accept a candidate draw. Larger values lead 
to closer approximations of the target distribution at the expense of longer runtimes. 


*args... : additional keyword arguments to be passed to the dist function. 
Value 

Returns a Sampler {ABCTune} type object. 
Example 


See the ABC Line and other Examples. 


ABCTune Type 


Declaration 


type ABCTune 
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Fields 


e datakeys: :Vector(Symbol) : stochastic “data” nodes in the full conditional distribution for parameters 
to be updated and nodes at which summary statistics are computed separately in the sampling algorithm. 


e Tsim: :Vector{Vector{Float64}} : simulated data summary statistics for the ns im data sets. 


e epsilon::Vector{Float 64} : adaptively tuned tolerances for the data sets. 


2.4.2 Adaptive Mixture Metropolis (AMM) 


Implementation of the Roberts and Rosenthal /75] adaptive (multivariate) mixture Metropolis /42]/43]/60] sampler 
for simulating autocorrelated draws from a distribution that can be specified up to a constant of proportionality. 


Model-Based Constructor 


AMM (params:: ElementOrVector(Symbol], Sigma: :Matrix{T<:Real}; adapt: :Symbol=:all) 
Construct a Sampler object for adaptive mixture Metropolis sampling. Parameters are assumed to be contin- 
uous, but may be constrained or unconstrained. 


Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


*Sigma : covariance matrix for the non-adaptive multivariate normal proposal distribution. The covariance 
matrix is relative to the unconstrained parameter space, where candidate draws are generated. 


*adapt [type of adaptation phase. Options are] 
— :all: adapt proposal during all iterations. 
— :burnin: adapt proposal during burn-in iterations. 
— :none : no adaptation (multivariate Metropolis sampling with fixed proposal). 
Value 
Returns a Sampler {AMMTune} type object. 
Example 


See the Seeds and other Examples. 


Stand-Alone Function 


amm! (v::AMMVariate, SigmaF::Cholesky{Float64}, logf:: Function; adapt::Bool=true) 
Simulate one draw from a target distribution using an adaptive mixture Metropolis sampler. Parameters are 
assumed to be continuous and unconstrained. 


Arguments 


*v : current state of parameters to be simulated. When running the sampler in adaptive mode, the v 
argument in a successive call to the function should contain the tune field returned by the previous call. 


*SigmaF : Cholesky factorization of the covariance matrix for the non-adaptive multivariate normal pro- 
posal distribution. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 
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*adapt : whether to adaptively update the proposal distribution. 


Value 


Returns v updated with simulated values and associated tuning parameters. 


Example 


The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. 


HMAAMAARA RARA RARA RARA RAAAMRARARARAMRAAARAAARAARARARARA RARA RARA HAHAHAHA 
## Linear Regression 

## y * N(bO + bl * x, s2) 

T4 b0, bi ~+ N(0, 1000) 

## s2 ~ invgamma(0.001, 0.001) 
KHAAA HEA AE EAH EPH AERA EERE aa aE Aaa AE AEE EE EE RE EAE 


using Mamba 
## Data 


data = Dict ( 
:x => [1, 2, 3, 4, 5], 


## Log-transformed Posterior(b0, bl, log(s2)) + Constant 


logf = function(x::DenseVector) 
b0 = x[1] 
bl = x[2] 


logs2 = x[3] 
r = data[:y] - b0 - bl x data[:x] 
(-0.5 + length(data[:y]) - 0.001) + logsz = 
(0.5 * dot(r, r) + 0.001) / exp(logs2) - 
0.5 « b0^2 / 1000 = 0.5 * b1^2 / 1000 
end 


## MCMC Simulation with Adaptive Multivariate Metopolis Sampling 
n = 5000 

burnin = 1000 

sim = Chains(n, 3, names = ["b0", "b1", "s2"]) 

theta = AMMVariate([0.0, 0.0, 0.0]) 

SigmaF = cholfact (eye (3) ) 

for i in 1:n 


amm! (theta, SigmaF, logf, adapt = (i <= burnin)) 
sim[i, :, 1] = [theta[1:2]; exp(theta[3]) ] 
end 


describe (sim) 


AMMVariate Type 


Declaration 


typealias AMMVariate SamplerVariate{AMMTune} 
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Fields 


e value::VectoríFloat64] : simulated values. 


* tune: : AMMTune : tuning parameters for the sampling algorithm. 


Constructors 


AMMVariate (x::AbstractVector{T<:Real}) 

AMMVariate (x::AbstractVector[T<:Real], tune::AMMTune) 
Construct a AMMVariate object that stores simulated values and tuning parameters for adaptive mixture 
Metropolis sampling. 


Arguments 

*x : simulated values. 

*tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a AMMVariate type object with fields set to the values supplied to arguments x and tune. 


AMMTune Type 


Declaration 


type AMMTune <: SamplerTune 


Fields 


e adapt: :Bool : whether the proposal distribution has been adaptively tuned. 


* beta::Real : proportion of weight given to draws from the non-adaptive proposal with covariance factor- 
ization SigmaF, relative to draws from the adaptively tuned proposal with covariance factorization SigmaLm, 
during adaptive updating. Fixed atbeta = 0.05. 


e m::Int : number of adaptive update iterations that have been performed. 


e Mv::Vector{Float 64} : running mean of draws v during adaptive updating. Used in the calculation of 
SigmaLm. 


e Mvv::Vector{Float64} : running mean of v * v’ during adaptive updating. Used in the calculation of 
SigmaLm. 


e scale::Real: fixed value 2.3872 inthe factor (scale / length(v))by which the adaptively updated 
covariance matrix is scaled—adopted from Gelman, Roberts, and Gilks /30/. 


e SigmaF: :Cholesky {Float 64} : factorization of the non-adaptive covariance matrix. 


e Sigmalm: :Matrix{Float64} : lower-triangular factorization of the adaptively tuned covariance matrix. 


2.4.3 Adaptive Metropolis within Gibbs (AMWG) 


Implementation of a Metropolis-within-Gibbs sampler /60][75][91] for iteratively simulating autocorrelated draws 
from a distribution that can be specified up to a constant of proportionality. 
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Model-Based Constructor 


AMWG (params::ElementOrVector[Symbol], sigma::ElementOrVector{T<:Real}; adapt::Symbol-:all, batch- 
size: :Integer-50, target: :Realz0.44) 
Construct a Sampler object for adaptive Metropolis-within-Gibbs sampling. Parameters are assumed to be 
continuous, but may be constrained or unconstrained. 


Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


*sigma : scaling value or vector of the same length as the combined elements of nodes params, defining 
initial standard deviations for univariate normal proposal distributions. Standard deviations are relative to 
the unconstrained parameter space, where candidate draws are generated. 


*adapt [type of adaptation phase. Options are] 
— :all: adapt proposals during all iterations. 
— :burnin: adapt proposals during burn-in iterations. 
— :none : no adaptation (Metropolis-within-Gibbs sampling with fixed proposals). 


*batchsize : number of samples that must be accumulated before applying an adaptive update to the 
proposal distributions. 


*target : target acceptance rate for the algorithm. 
Value 

Returns a Sampler {AMWGTune} type object. 
Example 


See the Birats, Blocker, and other Examples. 


Stand-Alone Function 


amwg ! (v::AMWGVariate, sigma:: Vector( T«:Real], logf:: Function; adapt::Bool=true, batchsize::Integer=50, 
target: : Realz0.44) 
Simulate one draw from a target distribution using an adaptive Metropolis-within-Gibbs sampler. Parameters 
are assumed to be continuous and unconstrained. 


Arguments 


*v : current state of parameters to be simulated. When running the sampler in adaptive mode, the v 
argument in a successive call to the function should contain the tune field returned by the previous call. 


*sigma : vector of the same length as v, defining initial standard deviations for univariate normal proposal 
distributions. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


*adapt : whether to adaptively update the proposal distribution. 


*batchsize: number of samples that must be newly accumulated before applying an adaptive update to 
the proposal distributions. 


*target : target acceptance rate for the adaptive algorithm. 


Value 


Returns v updated with simulated values and associated tuning parameters. 
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Example 


The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. Also, see the Line: Block- 
Specific Sampling with AMWG and Slice example. 


RAMAAARAAAAMRARA RAR AAA RAR ARAN AARAAAARAARAARANARAARANARAA AA RAR ARAN ANR AAN ARANA ARAN ANA 
## Linear Regression 

## y ce NBO + bi + x, 82) 

Ht b0, BI ~ N(O, 1000) 

Ht S2 ~ invgamma(0.001, 0.001) 
KARAHA 


using Mamba 


## Data 
data = Dict( 
2x => [1, 2, 3, 4, 5], 
sy => DL 3, Sp 2. 29] 
) 


## Log-transformed Posterior(b0, bl, log(s2)) + Constant 


logf = function(x::DenseVector) 
b0 = x[1] 
bl = x[2] 


logs2 = x[3] 
r = data[:y] - bO - bl x data[:x] 
(-0.5 » length(data[:y]) - 0.001) * logs2 - 
(0.5 * dot(r, r) + 0.001) / exp(logs2) - 
0.5. + b0^2 / 1000 = 0.5 x bl1^2 / 1000 
end 


## MCMC Simulation with Adaptive Metopolis-within-Gibbs Sampling 
n = 5000 
burnin - 1000 


sim = Chains(n, 3, names = ["b0", "pl", "s2"]) 
theta - AMWGVariate([0.0, 0.0, 0.0]) 
sigma = ones(3) 
for i in 1:n 
amwg!(theta, sigma, logf, adapt = (i <= burnin)) 
sim[i, :, 1] = [theta[1:2]; exp(theta[3])] 
end 


describe (sim) 


AMWGVariate Type 


Declaration 


typealias AMWGVariate SamplerVariate {AMWGTune} 


Fields 


e value: :Vector{Float64} : simulated values. 


* tune: : AMWGTune : tuning parameters for the sampling algorithm. 
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Constructors 


AMWGVariate (x::AbstractVector(T«:Real]) 

AMWGVariate (x::AbstractVector(T«:Real], tune::AMWGTune) 
Construct a AMWGVariate object that stores simulated values and tuning parameters for adaptive Metropolis- 
within-Gibbs sampling. 


Arguments 

*x : simulated values. 

“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a AMWGVariate type object with fields set to the values supplied to arguments x and tune. 


AMWGTune Type 


Declaration 


type AMWGTune «: SamplerTune 


Fields 


e adapt: :Bool : whether the proposal distribution has been adaptively tuned. 


e accept: : Vector(Int) : number of accepted candidate draws generated for each element of the parameter 
vector during adaptive updating. 


* batchsize::Int : number of samples that must be accumulated before applying an adaptive update to the 
proposal distributions. 


e m: :Int : number of adaptive update iterations that have been performed. 


e sigma: :VectoríFloat64] : updated values of the proposal standard deviations if adapt = true, and 
the user-defined values otherwise. 


e target: :Real : target acceptance rate for the adaptive algorithm. 


2.4.4 Binary Hamiltonian Monte Carlo (BHMC) 


Implementation of the binary-state Hamiltonian Monte Carlo sampler of Pakman /65]. The sampler simulates auto- 
correlated draws from a distribution that can be specified up to a constant of proportionality. 


Model-Based Constructor 


BHMC (params::ElementOrVector{Symbol}, traveltime::Real) 
Construct a Sampler object for BHMC sampling. Parameters are assumed to have binary numerical values (0 
or 1). 


Arguments 


*params : stochastic node(s) to be updated with the sampler. 
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*t raveltime : length of time over which particle paths are simulated. It is recommended that supplied 
values be of the form (n + i)m, where optimal choices of n € Z* are expected to grow with the parameter 
space dimensionality. 


Value 
Returns a Sampler { BHMCTune} type object. 
Example 


See the Pollution and other Examples. 


Stand-Alone Function 


bhmc! (v:: BHM CVariate, traveltime::Real, logf:: Function) 
Simulate one draw from a target distribution using the BHMC sampler. Parameters are assumed to have binary 
numerical values (0 or 1). 


Arguments 
ev : current state of parameters to be simulated. 


*traveltime : length of time over which particle paths are simulated. It is recommended that supplied 
values be of the form (n + im, where optimal choices of n € Z* are expected to grow with the parameter 
space dimensionality. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


Value 
Returns v updated with simulated values and associated tuning parameters. 


Example 


HAHAHAHA 
## Linear Regression 

## y ~ MvNormal(X * (beta0 .* gamma), 1) 
## gamma ~ DiscreteUniform(0, 1) 
HHH HAHAHAHA 


using Mamba 


## Data 

n, p = 25, 10 

X = randn(n, p) 

beta0 = randn (p) 

gamma0 = rand(0:1, p) 

y = X x (beta0 .* gamma0) + randn (n) 


## Log-transformed Posterior(gamma) + Constant 


logf = function (gamma : :DenseVector) 
logpdf (MvNormal (X * (beta0 .* gamma), 1.0), y) 
end 


## MCMC Simulation with Binary Hamiltonian Monte Carlo Sampling 
t = 10000 
sim = Chains(t, p, names = map(i -> "gamma[$i]", 1:p)) 
gamma = BHMCVariate (zeros (p) ) 
for i in l;t 
bhmc! (gamma, (2 * p + 0.5) * pi, logf) 
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sim[i, :, 1] = gamma 
end 
describe (sim) 


BHMCVariate Type 


Declaration 


typealias BHMCVariate SamplerVariate(BHMCTune] 


Fields 


e value::VectoríFloat64] : simulated values. 


e tune: : BHMCTune : tuning parameters for the sampling algorithm. 


Constructors 


BHMCVariate (x::AbstractVector(T«:Real]) 
BHMCVariate (x::AbstractVector[T<:Real], tune:: BHMCTune) 
Construct a BHMCVariate object that stores simulated values and tuning parameters for BHMC sampling. 


Arguments 

*x : simulated values. 

“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a BHMCVariate type object with fields set to the values supplied to arguments x and tune. 
BHMCTune Type 
Declaration 


type BHMCTune «: SamplerTune 


Fields 


e traveltime::Float64: length of time over which particle paths are simulated. 
e position: :Vector(Float64) : initial particle positions. 


e velocity: :Vector{Float 64} : initial particle velocites. 


e wallhits:: Int : number of times particles are reflected off the 0 threshold. 


e wallcrosses: : Int : number of times particles travel through the threshold. 
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2.4.5 Binary MCMC Model Composition (BMC3) 


Implementation of the binary-state MCMC Model Composition of Madigan and York /55/ in which proposed updates 
are always state changes. Liu /52/ shows this sampler is more efficient than Gibbs sampling for a binary vector. 
Schafer [79][80] proposes a method for block updates of binary vectors using this sampler. The sampler simulates 
autocorrelated draws from a distribution that can be specified up to a constant of proportionality. 


Model-Based Constructors 


BMC3 (params::ElementOrVector[Symbol); k::Integer=1) 

BMC3 (params: :ElementOrVector{Symbol}, indexset:: Vector{ Vector{Int}}) 
Construct a Sampler object for BMC3 sampling. Parameters are assumed to have binary numerical values (0 
or 1). 


Arguments 
*params : stochastic node(s) to be updated with the sampler. 
ek : number of parameters to select at random for simultaneous updating in each call of the sampler. 
*indexset : candidate set of indices of the parameters whose states are to be changed simultaneously. 
Value 


Returns a Sampler {BMC3Tune} type object. 


Stand-Alone Functions 


bmc3! (v:: BMMGvVariate, logf:: Function; k::Integer=1) 

bmc3! (v::BMMGVariate, indexset:: Vector[ Vector[Int]], logf:: Function) 
Simulate one draw from a target distribution using the BMC3 sampler. Parameters are assumed to have binary 
numerical values (0 or 1). 


Arguments 
ev : current state of parameters to be simulated. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


ek : number of parameters, such that k <= length (v), to select at random for simultaneous updating 
in each call of the sampler. 


*indexset : candidate set of indices of the parameters whose states are to be changed simultaneously. 
Value 
Returns v updated with simulated values and associated tuning parameters. 


Example 


HAAAAAAAAAARARARARAARARARARARAARARAARARAARARARA RARA RARA ARAARARAAAARARARAAAARARA RARA RA 
## Linear Regression 

## y ~ MvNormal (X * (beta0 .* gamma), 1) 
## gamma ~ DiscreteUniform(0, 1) 


TAAAAAAAA AAA RARA RARA RARA RARA RARA RARA RARA RARA RARA RARA AAAARA DARA HAAA 


using Mamba 


## Data 
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n, p = 25, 10 

X = randn(n, p) 

beta0 = randn (p) 

gamma0 = rand(0:1, p) 

y = X x (beta0 .* gamma0) + randn (n) 


## Log-transformed Posterior(gamma) + Constant 


logf = function (gamma: :DenseVector) 
logpdf(MvNormal(X * (beta0 .* gamma), 1.0), y) 
end 


## MCMC Simulation with Binary MCMC Model Composition 
t = 10000 
sim = Chains(t, p, names = map(i -> "gamma[$i]", 1:p)) 
gamma = BMC3Variate (zeros (p)) 
for i in l:t 
bmc3! (gamma, logf) 
sim[i, :, 1] = gamma 
end 
describe (sim) 


p = plot(sim, [:trace, :mixeddensity]) 
draw(p, filename = "bmc3plot") 
BMC3Variate Type 
Declaration 


typealias BMC3Variate SamplerVariate{BMC3Tune} 


Fields 


e value: :Vector{Float64} : simulated values. 


e tune: :BMC3Tune: tuning parameters for the sampling algorithm. 


Constructors 


BMC3Variate (x::AbstractVector(T«:Real]) 
BMC3Variate (x::AbstractVector[T<:Real], tune:: BMC3Tune) 
Construct a BMC3Variate object that stores simulated values and tuning parameters for BMC3 sampling. 


Arguments 

*x : simulated values. 

“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a BMC3Variate type object with fields set to the values supplied to arguments x and tune. 
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BMC3Tune Type 


Declaration 


type BMC3Tune «: SamplerTune 


Fields 


e indexset::Vector{Vector{Int}} : candidate set of indices of the parameters whose states are to be 
changed simultaneously. 


2.4.6 Binary Metropolised Gibbs (BMG) 


Implementation of the binary-state Metropolised Gibbs sampler described by Schafer /79]/80] in which components 
are drawn sequentially from full conditional marginal distributions and accepted together in a single Metropolis- 
Hastings step. The sampler simulates autocorrelated draws from a distribution that can be specified up to a constant of 
proportionality. 


Model-Based Constructor 


BMG (params: :ElementOrVector{Symbol}; k::Integer=1) 
Construct a Sampler object for BMG sampling. Parameters are assumed to have binary numerical values (0 
or 1). 


Arguments 
*params : stochastic node(s) to be updated with the sampler. 
*k : number of parameters to select at random for simultaneous updating in each call of the sampler. 
Value 
Returns a Sampler {BMGTune} type object. 
Example 


See the Pollution and other Examples. 


Stand-Alone Function 


bmg! (v::BMGvVariate, logf:: Function; k::Integer=1) 
Simulate one draw from a target distribution using the BMG sampler. Parameters are assumed to have binary 
numerical values (0 or 1). 


Arguments 
ev : current state of parameters to be simulated. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


ek : number of parameters, such that k <= length (v), to select at random for simultaneous updating 
in each call of the sampler. 


Value 


Returns v updated with simulated values and associated tuning parameters. 


2.4. Sampling Functions 67 


Mamba.jl Documentation, Release 0.8.2 


Example 


KHAAA HAHAHAHA HAHAHAHA 


## Linear Regression 
## y ~ MvNormal(X * (beta0 .* gamma), 1) 
HA gamma ~ DiscreteUniform(0, 1) 


TAAAAARAMA RARA RA RARA RARA TADA RARA RARA RARA RARA RARA RARA AAAARA DARA RARA RARA RADA RATA 


using Mamba 


## Data 

n, p = 25, 10 

X = randn(n, p) 

beta0 = randn (p) 

gamma0 = rand(0:1, p) 

y = X x (beta0 .* gamma0) + randn (n) 


## Log-transformed Posterior(gamma) + Constant 


logf = function (gamma: :DenseVector) 
logpdf(MvNormal(X * (beta0 .* gamma), 1.0), y) 
end 


## MCMC Simulation with Binary Metropolised Gibbs 
t = 10000 
sim = Chains(t, p, names = map(i -> "gamma[Si]", 
gamma = BMGVariate (zeros (p) ) 
for 1 in l:t 
bmg! (gamma, logf) 
sim[i, :, 1] = gamma 
end 
describe (sim) 


1:p)) 


BMGVariate Type 


Declaration 


typealias BMGVariate SamplerVariate{BMGTune} 


Fields 


e value: :Vector{Float64} : simulated values. 


e tune: :BMGTune: tuning parameters for the sampling algorithm. 


Constructors 


BMGVariate (x::AbstractVector(T«:Real]) 
BMGVariate (x::AbstractVector[T<:Real], tune:: BMGTune) 


Construct a BMGVariate object that stores simulated values and tuning parameters for BMG sampling. 


Arguments 


*x : simulated values. 


“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
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Value 


Returns a BMGVariate type object with fields set to the values supplied to arguments x and tune. 


BMGTune Type 


Declaration 


type BMGTune «: SamplerTune 


2.4.7 Discrete Gibbs Sampler (DGS) 


Implementation of a sampler for the simulation of discrete or discretized model parameters with finite support. Draws 
are simulated directly from a probability mass function that can be specified up to a constant of proportionality. Note 
that versions of this sampler evaluate the probability function over all points in the parameter space; and, as a result, 
may be very computationally intensive for large spaces. 


Model-Based Constructor 


DGS (params::ElementOrVector[Symbol]) 
Construct a Sampler object for which DGS sampling is to be applied separately to each of the supplied 
parameters. Parameters are assumed to have discrete univariate distributions with finite supports. 


Arguments 
*params : stochastic node(s) to be updated with the sampler. 
Value 
Returns a Sampler(DGSTune] type object. 
Example 


See the Eyes, Pollution, and other Examples. 


Stand-Alone Functions 


dgs! (v::DGSVariate, support: :Matrix[ T«: Real], logf:: Function) 

dgs! (v::DGSVariate, support: :Matrix[ T«:Real], probs::Vector[Float64] ) 
Simulate one draw directly from a target probability mass function. Parameters are assumed to have discrete 
and finite support. 


Arguments 
ev : current state of parameters to be simulated. 


“support : matrix whose columns contain the vector coordinates in the parameter space from which to 
simulate values. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


*probs : sampling probabilities for the columns of support. 
Value 


Returns v updated with simulated values and associated tuning parameters. 
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DGSVariate Type 


Declaration 


typealias DGSVariate SamplerVariate{DGSTune} 


Fields 


e value::VectoríFloat64] : simulated values. 


e tune: :DGSTune : tuning parameters for the sampling algorithm. 


Constructors 


DGSVariate (x::AbstractVector(T«:Real]) 
DGSVariate (x::AbstractVector(T«:Real], tune:: DGSTune) 
Construct a DGSVariate object that stores simulated values and tuning parameters for DGS sampling. 


Arguments 

*x : simulated values. 

“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a DGSVariate type object with fields set to the values supplied to arguments x and tune. 
DGSTune Type 
Declaration 


type DGSTune «: SamplerTune 


Fields 


e support: :Matrix{Real} : matrix whose columns contain the vector coordinates in the parameter space 
from which to simulate values. 


e probs::Vector{Float64} : sampling probabilities for the columns of support. 


2.4.8 Hamiltonian Monte Carlo (HMC) 


Implementation of the Hybrid Monte Carlo (also known as Hamiltonian Monte Carlo) of Duane /2//. The sampler 
simulates autocorrelated draws from a distribution that can be specified up to a constant of proportionality. Code is 
derived from Neal's implementation /62]. 
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Model-Based Constructors 


HMC (params::ElementOrVector{Symbol}, epsilon: :Real, L::Integer; dtype::Symbol=:forward) 

HMC (params: :ElementOrVector{Symbol}, epsilon: :Real, L::Integer, Sigma: :Matrix{T<:Real}; 
dtype::Symbol=: forward ) 
Construct a Sampler object for HMC sampling. Parameters are assumed to be continuous, but may be con- 
strained or unconstrained. 


Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


*epsilon: step size. 
eL : number of steps to take in the Leapfrog algorithm. 


*Sigma : covariance matrix for the multivariate normal proposal distribution. The covariance matrix is 


relative to the unconstrained parameter space, where candidate draws are generated. If omitted, the identity 
matrix is assumed. 


*dtype [type of differentiation for gradient calculations. Options are] 
— :central: central differencing. 
— : forward: forward differencing. 
Value 
Returns a Sampler { HMCTune} type object. 
Example 


See the Dyes and other Examples. 


Stand-Alone Functions 


hmc! (v::HMCVariate, epsilon::Real, L::Integer, logfgrad::Function) 
hmc! (v::HMCVariate, epsilon::Real, L::Integer, SigmaF::Cholesky{Float64}, logfgrad:: Function) 


Simulate one draw from a target distribution using the HMC sampler. Parameters are assumed to be continuous 
and unconstrained. 


Arguments 
ev : current state of parameters to be simulated. 
*epsilon: step size. 
eL : number of steps to take in the Leapfrog algorithm. 


*SigmaF : Cholesky factorization of the covariance matrix for the multivariate normal proposal distribu- 
tion. If omitted, the identity matrix is assumed. 


*logfgrad : function that takes a single DenseVector argument at which to compute the log- 


transformed density (up to a normalizing constant) and gradient vector, and returns the respective results 
as a tuple. 


Value 


Returns v updated with simulated values and associated tuning parameters. 


Example 
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The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. 


KARAHA RARA RAR AMARA HHH 
## Linear Regression 

## y ~ N(b0 + bl + x, s2) 

## b0, bl ~ N(O, 1000) 

Ht S2 + invgamma(0.001, 0.001) 
KHAAA HHH 


using Mamba 


## Data 
data = Dict( 
:x => [1, 2, 3, 4, 5], 
¿y => [1, 3, 3, 3, 5] 
) 


## Log-transformed Posterior(b0, bl, log(s2)) + Constant and Gradient Vector 
logfgrad = function(x::DenseVector) 


b0 = x[1] 

b1 = x[2] 

logs2 = x[3] 

r = data[:y] - bO - bl x data[:x] 


logf = (-0.5 * length(data[:y]) - 0.001) x* logs2 - 
(0.5 * dot(r, r) + 0.001) / exp(logs2) - 
0.5- x 5082. / 1000 — 0:5 + b1^2 / 1000 
grad = [ 
sum(r) / exp(logs2) - b0 / 1000, 
sum(data[:x] .* r) / exp(logs2) - bl / 1000, 
-0.5 « length(data[:y]) - 0.001 + (0.5 * dot (r, r) + 0.001) / exp(logs2) 
] 
logf, grad 
end 


## MCMC Simulation with Hamiltonian Monte Carlo 

## Without (1) and with (2) a user-specified proposal covariance matrix 

n = 5000 

siml = Chains(n, 3, names = ["b0", "pi", "s2"]) 

sim2 = Chains(n, 3, names = ["b0", "b1", "s2"]) 

thetal = HMCVariate([0.0, 0.0, 0.0]) 

theta2 - HMCVariate([0.0, 0.0 

epsilon = 0.1 

L = 50 

SigmaF = cholfact (eye (3) ) 

for i in 1:n 
hmc!(thetal, epsilon, L, logfgrad) 
hmc!(theta2, epsilon, L, SigmaF, logfgrad) 
siml[i, :, 1] [thetal[1:2]; exp(thetal[3]) ] 
sim2[i, :, 1] = [theta2[1:2]; exp(theta2[3])] 

end 

describe (sim1) 

describe (sim2) 
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HMCVariate Type 


Declaration 


typealias HMCVariate SamplerVariate{HMCTune} 


Fields 


e value::VectoríFloat64] : simulated values. 


* tune: : HMCTune : tuning parameters for the sampling algorithm. 


Constructors 


HMCVariate (x::AbstractVector[T<:Real]) 
HMCVariate (x::AbstractVector(T«:Real], tune:: HMCTune) 
Construct a HMCVariate object that stores simulated values and tuning parameters for HMC sampling. 


Arguments 

*x : simulated values. 

“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a HMCVariate type object with fields set to the values supplied to arguments x and tune. 


HMCTune Type 


Declaration 


type HMCTune <: SamplerTune 


Fields 


e epsilon::Float64:step size. 
e L::Int : number of steps to take in the Leapfrog algorithm. 


e SigmaF: :Cholesky(Float64) : Cholesky factorization of the covariance matrix for the multivariate nor- 
mal proposal distribution. 


2.4.9 Metropolis-Adjusted Langevin Algorithm (MALA) 


Implementation of the Metropolis-Adjusted Langevin Algorithm (MALA) of Roberts and Tweedie /77] and Roberts 
and Stramer /76]. The sampler simulates autocorrelated draws from a distribution that can be specified up to a constant 
of proportionality. MALA is related to Hamiltonian Monte Carlo as described thoroughly by Girolami and Calderhead 
[38]. 
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Model-Based Constructors 


MALA (params::ElementOrVector{Symbol}, scale::Real; dtype::Symbol=:forward) 

MALA (params::ElementOrVector[Symbol], scale::Real, Sigma::Matrix{T<:Real}; dtype::Symbol=:forward) 
Construct a Sampler object for MALA sampling. Parameters are assumed to be continuous, but may be 
constrained or unconstrained. 


Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


*scale : factor by which the drift and covariance matrix of the proposal distribution are scaled. 


*Sigma : covariance matrix for the multivariate normal proposal distribution. The covariance matrix is 
relative to the unconstrained parameter space, where candidate draws are generated. If omitted, the identity 
matrix is assumed. 


*dtype [type of differentiation for gradient calculations. Options are] 
— :central: central differencing. 
— : forward: forward differencing. 
Value 
Returns a Sampler {MALATune} type object. 
Example 


See the Dyes and other Examples. 


Stand-Alone Functions 


mala! (v::MALAVariate, scale::Real, logfgrad:: Function) 

mala! (v::MALAVariate, scale::Real, SigmaF::Cholesky[Float64), logfgrad::Function) 
Simulate one draw from a target distribution using the MALA sampler. Parameters are assumed to be continuous 
and unconstrained. 


Arguments 
ev : current state of parameters to be simulated. 
escale : factor by which the drift and covariance matrix of the proposal distribution are scaled. 


*SigmaF : Cholesky factorization of the covariance matrix for the multivariate normal proposal distribu- 
tion. If omitted, the identity matrix is assumed. 


*logfgrad: function that takes a single DenseVector argument of parameter values at which to com- 
pute the log-transformed density (up to a normalizing constant) and gradient vector, and returns the re- 
spective results as a tuple. 


Value 
Returns v updated with simulated values and associated tuning parameters. 
Example 


The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. 
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KARAHA HHH HHH 
## Linear Regression 

## y * N(b0 + bl * x, s2) 

## b0, bl ~ N(O, 1000) 

Ht s2 ~ invgamma(0.001, 0.001) 
IMAMAMAA RADAR ARAN AA RAR ARANA DARAN ARANA HHH 


using Mamba 


## Data 
data = Dict( 
sx => [ly 2, 3, 4, 51, 
sy => [1, 3, 3, 3, 5] 
) 


## Log-transformed Posterior(b0, bl, log(s2)) + Constant and Gradient Vector 
logfgrad = function (x::DenseVector) 


b0 = x[1] 

b1 = x[2] 

logs2 = x[3] 

r = data[:y] - b0 - bl x data[:x] 

logf = (-0.5 * length(data[:y]) - 0.001) x* logs2 - 
(0.5 * dot(r, r) + 0.001) / exp(logs2) - 
0.5. x 5082: / 1000 = 0.5.4 bi1^2 / 1000 

grad [ 


sum(r) / exp(logs2) - b0 / 1000, 
sum(data[:x] .* r) / exp(logs2) - bl / 1000, 
-0.5 « length(data[:y]) - 0.001 + (0.5 * dot(r, r) + 0.001) / exp(logs2) 
] 
logf, grad 
end 


## MCMC Simulation with Metropolis-Adjusted Langevin Algorithm 
## Without (1) and with (2) a user-specified proposal covariance matrix 


n = 5000 
siml = Chains(n, 3, names = ["b0", "b1", "s2"]) 
sim2 = Chains(n, 3, names = ["b0", "b1", "s2"]) 


thetal - MALAVariate([0.0, 0.0, 0.0]) 
theta2 = MALAVariate([0.0, 0.0, 0.0]) 
scale = 0.1 
SigmaF = cholfact (eye(3)) 
for i in 1:n 
mala!(thetal, scale, logfgrad) 
mala!(theta2, scale, SigmaF, logfgrad) 


siml[i, :, 1] = [thetal[1:2]; exp(thetal[3])] 
sim2[i, :, 1] = [theta2[1:2]; exp(theta2[3])] 
end 


describe(siml) 
describe(sim2) 


MALAVariate Type 


Declaration 


typealias MALAVariate SamplerVariate(MALATune] 
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Fields 


e value::Vector{Float64} : simulated values. 


* tune: :MALATune : tuning parameters for the sampling algorithm. 


Constructors 


MALAVariate (x::AbstractVector(T«:Real]) 
MALAVariate (x::AbstractVector(T«:Real], tune:: MALATune) 
Construct a MALAVariate object that stores simulated values and tuning parameters for MALA sampling. 


Arguments 

*x : simulated values. 

*tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a MALAVariate type object with fields set to the values supplied to arguments x and tune. 


MALATune Type 


Declaration 


type MALATune «: SamplerTune 


Fields 


e scale::Float64 : factor by which the drift and covariance matrix of the proposal distribution are scaled. 


e SigmaF: :Cholesky(íFloat64) : Cholesky factorization of the covariance matrix for the multivariate nor- 
mal proposal distribution. 


2.4.10 Missing Values Sampler (MISS) 


A sampler to simulate missing output values from their likelihood distributions. 


Model-Based Constructor 


MISS (params::ElementOrVector{Symbol}) 
Construct a Sampler object to sampling missing output values. The constructor should only be used to sample 
stochastic nodes upon which no other stochastic node depends. So-called ‘output nodes’ can be identified with 
the keys () function. Moreover, when the MISS constructor is included in a vector of Sampler objects to 
define a sampling scheme, it should be positioned at the beginning of the vector. This ensures that missing 
output values are updated before any other samplers are executed. 


Arguments 
*params : stochastic node(s) that contain missing values (NaN) to be updated with the sampler. 
Value 


Returns a Sampler{Dict{Symbol, MISSTune} } type object. 
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Example 


See the Bones and other Examples. 


MISSTune Type 


Declaration 


type MISSTune 


Fields 


e dims: :Tuple : dimensions of a stochastic node to be updated. 
e valueinds::Vector(Int] : indices to missing values in the node. 


e distrinds::Vector{Int} : indices to node distributions from which to sample the missing values. 


2.4.11 No-U-Turn Sampler (NUTS) 


Implementation of the NUTS extension (algorithm 6) /45/ to Hamiltonian Monte Carlo /62/ for simulating autocor- 
related draws from a distribution that can be specified up to a constant of proportionality. 


Model-Based Constructor 


NUTS (params::ElementOrVector{Symbol}; dtype::Symbol=:forward, target::Real=0.6 ) 
Construct a Sampler object for No-U-Turn sampling, with the algorithm’s step size parameter adaptively tuned 
during burn-in iterations. Parameters are assumed to be continuous, but may be constrained or unconstrained. 


Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


*dtype [type of differentiation for gradient calculations. Options are] 
— :central: central differencing. 
— : forward: forward differencing. 
*target : target acceptance rate for the algorithm. 
Value 
Returns a Sampler {NUTSTune} type object. 
Example 


See the Dyes, Equiv, and other Examples. 


Stand-Alone Functions 


nutsepsilon (v::NUTSVariate, logfgrad:: Function) 
Generate an initial value for the step size parameter of the No-U-Turn sampler. Parameters are assumed to be 
continuous and unconstrained. 


Arguments 
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ev : the current state of parameters to be simulated. 


*logfgrad: function that takes a single DenseVector argument of parameter values at which to com- 
pute the log-transformed density (up to a normalizing constant) and gradient vector, and returns the re- 
spective results as a tuple. 


Value 
A numeric step size value. 


nuts! (v::NUTSVariate, epsilon::Real, logfgrad:: Function; adapt::Bool=false, target: :Real-0.6) 
Simulate one draw from a target distribution using the No-U-Turn sampler. Parameters are assumed to be 
continuous and unconstrained. 


Arguments 


*v : current state of parameters to be simulated. When running the sampler in adaptive mode, the v 
argument in a successive call to the function should contain the tune field returned by the previous call. 


*epsilon: the NUTS algorithm step size parameter. 


*logfgrad : function that takes a single DenseVector argument at which to compute the log- 
transformed density (up to a normalizing constant) and gradient vector, and returns the respective results 
as a tuple. 


*adapt : whether to adaptively update the epsilon step size parameter. 
*target : target acceptance rate for the algorithm. 
Value 
Returns v updated with simulated values and associated tuning parameters. 
Example 


The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. 


HAMAARARA RARA RARA RAR ARAAAAAAARARAMRAAMARAAARARARARARA RARA RARA RARA RARA RAR ARAA RARA NA 
## Linear Regression 

## y ~ N(b0 + bl xx, s2) 

## b0, bl ~ N(0, 1000) 

## s2 ~ invgamma(0.001, 0.001) 
HAAARARA RARA RARA RARA RARAMRARARARARAAMAARARARARARARARA RARA RARA deed deed deed dee dtd eat 


using Mamba 


## Data 
data = Dict( 
sx => [1, 2, 3, 4, 5], 
¿y => [1, 3, 3, 3, 5] 
) 


## Log-transformed Posterior(b0, bl, log(s2)) + Constant and Gradient Vector 
logfgrad = function (x::DenseVector) 


bO = x[1] 

bl = x[2] 

logs2 = x[3] 

r = data[:y] - b0 - bl x* data[:x] 

logf = (-0.5 * length(data[:y]) - 0.001) * logs2 - 
(0.5 * dot(r, r) + 0.001) / exp(logs2) - 
0.5 * b0^2 / 1000 = 0.5 + b1^2 / 1000 

grad = [ 
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sum(r) / exp(logs2) - b0 / 1000, 

sum(data[:x] .* r) / exp(logs2) - bl / 1000, 

-0.5 « length(data[:y]) - 0.001 + (0.5 * dot(r, r) + 0.001) / exp(logs2) 
] 
logf, grad 


end 


## MCMC Simulation with No-U-Turn Sampling 

n = 5000 

burnin = 1000 

sim = Chains(n, 3, start = (burnin + 1), names = ["b0", "b1", "s2"]) 
theta = NUTSVariate([0.0, 0.0, 0.0]) 

epsilon - nutsepsilon(theta, logfgrad) 

for i in 1:n 


nuts! (theta, epsilon, logfgrad, adapt = (i <= burnin)) 
if i > burnin 
sim[i, :, 1] = [theta[1:2]; exp(theta[3]) ] 
end 
end 


describe (sim) 


NUTSVariate Type 


Declaration 


typealias NUTSVariate SamplerVariate{NUTSTune} 


Fields 


* value::Vector{Float64} : simulated values. 


* tune: :NUTSTune : tuning parameters for the sampling algorithm. 


Constructors 


NUTSVariate (x::AbstractVector(T«:Real]) 
NUTSVariate (x-:AbstractVector(T«:Real], tune::NUTSTune) 
Construct a NUTSVariate object that stores simulated values and tuning parameters for No-U-Turn sampling. 


Arguments 

*x : simulated values. 

“tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a NUTSVariate type object with fields set to the values supplied to arguments x and tune. 
NUTSTune Type 
Declaration 


type NUTSTune «: SamplerTune 
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Fields 


adapt::Bool : whether the proposal distribution has been adaptively tuned. 


alpha: :Float64: cumulative acceptance probabilities o from leapfrog steps. 


epsilon::Float64 : updated value of the step size parameter €m = exp (u — vmHs f») if adapt - 
true, and the user-defined value otherwise. 


epsbar::Float64 i dual averaging parameter, defined as Em = 
exp (m”* log(€m) + (1 — m~") log(Em-—1)). 


gamma: : F1oat64 : dual averaging parameter, fixed at y = 0.05. 


Hbar::Float64: dual averaging parameter, defied as Hm = (1 = zi) Hac m (target 2). 
kappa: :Float64 : dual averaging parameter, fixed at & = 0.05. 

m: : Int : number of adaptive update iterations m that have been performed. 

mu: :Float 64 : dual averaging parameter, defined as u = log(10€p). 


nalpha: : Int : the total number na of leapfrog steps performed. 


t0::Float 64 : dual averaging parameter, fixed at to = 10. 


target::Float 64: target acceptance rate for the adaptive algorithm. 


2.4.12 Random Walk Metropolis (RWM) 


Random walk Metropolis-Hastings algorithm /43//60] in which parameters are sampled from symmetric distribu- 
tions centered around the current values. The sampler simulates autocorrelated draws from a distribution that can be 
specified up to a constant of proportionality. 


Model-Based Constructor 


RWM (params: :ElementOrVector{Symbol}, scale: :ElementOrVector{T<:Real}; pro- 


posal: :SymDistributionType=Normal) 
Construct a Sampler object for RWM sampling. Parameters are assumed to be continuous, but may be con- 


strained or unconstrained. 
Arguments 


*params : stochastic node(s) to be updated with the sampler. Constrained parameters are mapped to 
unconstrained space according to transformations defined by the Stochastic unlist () function. 


escale : scaling value or vector of the same length as the combined elements of nodes params for the 
proposal distribution. Values are relative to the unconstrained parameter space, where candidate draws 
are generated. 


“proposal : symmetric distribution of type Biweight, Cosine, Epanechnikov, Normal, 
SymTriangularDist, SymUniform, or Triweight to be centered around current parameter val- 
ues and used to generate proposal draws. Specified scale determines the standard deviations of Normal 
proposals and widths of the others. 


Value 
Returns a Sampler {RWMTune} type object. 


Example 
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See the Dyes and other Examples. 


Stand-Alone Function 


rwm! (v::RWMVariate, scale: :ElementOrVector{T<:Real}, logf:: Function; pro- 


posal: :SymDistributionType=Normal) 
Simulate one draw from a target distribution using an RWM sampler. Parameters are assumed to be continuous 


and unconstrained. 
Arguments 
ev : current state of parameters to be simulated. 
*scale: scalar or vector of the same length as v for the proposal distribution. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


*proposal : symmetric distribution of type Biweight, Cosine, Epanechnikov, Normal, 
SymTriangularDist, SymUniform, or Triweight to be centered around current parameter val- 
ues and used to generate proposal draws. Specified scale determines the standard deviations of Normal 
proposals and widths of the others. 


Value 
Returns v updated with simulated values and associated tuning parameters. 
Example 


The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. 


HAMAAMAARA RARA RARA HAHAHAHA AHAAA 
## Linear Regression 

## y ~ N(bO + bl * x, s2) 

## b0, bl ~ N(0, 1000) 

## s2 ~ invgamma(0.001, 0.001) 
HMAAAARA RARA RARA RAR ARARARARARARARAAAARARARARARARARA RARA RARA RAR ARA RA RAR ARAAARRA NA 


using Mamba 


## Data 
data = Dict( 
sx => [1, 2, 3, 4, 5], 
sy => [1, 3, 3, 3, 5] 
) 


## Log-transformed Posterior(b0, bi, log(s2)) + Constant 


logf = function (x::DenseVector) 
b0 = x[1] 
bl = x[2] 


logs2 = x[3] 
r = data[:y] - bO - bl * data[:x] 
(-0.5 » length(data[:y]) - 0.001) * logs2 - 
(0.5 * dot(r, r) + 0.001) / exp(logs2) = 
0.5 = b0^2 / 1000 = 0.5 * b1^2 / 1000 
end 


## MCMC Simulation with Random Walk Metropolis 
n = 5000 
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burnin = 1000 


sim = Chains(n, 3, names = ["b0", "b1", "s2"]) 
theta = RWMVariate([0.0, 0.0, 0.0]) 
scale = [0.5, 0.25, 1.0] 


for i in 1:n 
rwm!(theta, scale, logf, proposal-SymUniform) 
sim[i, :, 1] = [theta[1:2]; exp(theta[3])] 
end 
describe (sim) 


RWMvVariate Type 


Declaration 


typealias RWMVariate SamplerVariate{RWMTune} 


Fields 


e value: :Vector{Float64} : simulated values. 


* tune: : RWMTune : tuning parameters for the sampling algorithm. 


Constructors 


RWMVariate (x::AbstractVector[T<:Real]) 
RWMVariate (x::AbstractVector[T«:Real], tune:: RWMTune) 
Construct a RWMVariate object that stores simulated values and tuning parameters for RWM sampling. 


Arguments 

*x : simulated values. 

*tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a RWMVariate type object with fields set to the values supplied to arguments x and tune. 


RWMTune Type 


Declaration 


type RWMTune «:  SamplerTune 


Fields 


e scale::Union{Real, Vector}: scaling for the proposal distribution. 


* proposal::SymDistributionType : proposal distribution. 
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2.4.13 Shrinkage Slice (Slice) 


Implementation of the shrinkage slice sampler of Neal /6// for simulating autocorrelated draws from a distribution 
that can be specified up to a constant of proportionality. 


Model-Based Constructor 


Slice (params::ElementOrVector{Symbol}, width::ElementOrVector{T<:Real}, stype::Symbol-:multivar; 


transform: :Bool=false ) 
Construct a Sampler object for shrinkage slice sampling. Parameters are assumed to be continuous, but may 


be constrained or unconstrained. 
Arguments 
*params : stochastic node(s) to be updated with the sampler. 


“width: scaling value or vector of the same length as the combined elements of nodes params, defining 
initial widths of a hyperrectangle from which to simulate values. 


*stype [sampler type. Options are] 
— :multivar : Joint multivariate sampling of parameters. 
— :univar : Sequential univariate sampling. 


*transform: whether to sample parameters on the link-transformed scale (unconstrained parameter 
space). If true, then constrained parameters are mapped to unconstrained space according to transfor- 
mations defined by the Stochastic unlist () function, and width is interpreted as being relative to the 
unconstrained parameter space. Otherwise, sampling is relative to the untransformed space. 


Value 
Returns a Sampler(SliceTune] type object. 
Example 


See the Birats, Rats, and other Examples. 


Stand-Alone Function 


slice! (v::SliceVariate, width::ElementOrVector{T<:Real}, logf:: Function, stype::Symbol=:multivar ) 
Simulate one draw from a target distribution using a shrinkage slice sampler. Parameters are assumed to be 
continuous, but may be constrained or unconstrained. 


Arguments 
evy : current state of parameters to be simulated. 


ewidth : scalar or vector of the same length as v, defining initial widths of a hyperrectangle from which 
to simulate values. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 
the log-transformed density (up to a normalizing constant). 


*stype [sampler type. Options are] 
— :multivar : Joint multivariate sampling of parameters. 
— :univar : Sequential univariate sampling. 
Value 


Returns v updated with simulated values and associated tuning parameters. 
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Example 


The following example samples parameters in a simple linear regression model. Details of the model 
specification and posterior distribution can be found in the Supplement. Also, see the Line: Block- 
Specific Sampling with AMWG and Slice example. 


RAMAAARMAA RARA RARA NAAA RAR ARAN AARAAAARAARAARANARAARARARAAAARAR ARAN AN AAN ARANA ARAN ANA 
## Linear Regression 

## y ce NBO + bi + x, 82) 

Ht b0, BI ~ N(O, 1000) 

Ht S2 ~ invgamma(0.001, 0.001) 
KARAHA 


using Mamba 


## Data 
data = Dict( 
2x => [1, 2, 3, 4, 5], 
sy => UL, 3, Sy 3. 2] 
) 


## Log-transformed Posterior(b0, bl, log(s2)) + Constant 


logf = function(x::DenseVector) 
b0 = x[1] 
bl = x[2] 


logs2 = x[3] 
r = data[:y] - b0 - bl x data[:x] 
(-0.5 » length(data[:y]) - 0.001) * logs2 - 
(0.5 * dot(r, r) + 0.001) / exp(logs2) - 
0.5. + bU0^2 / 1000 = 0:5 x DIZ / 1000 
end 


## MCMC Simulation with Slice Sampling 
## With multivariate (1) and univariate (2) updating 


n = 5000 
siml = Chains(n, 3, names = ["b0", "bi", "s2"]) 
sim2 = Chains(n, 3, names = 


p 0207) 
r 0.01) 


[ "15 
["bO0", "5I", "s2"]) 
thetal = SliceVariate([0.0, 0.0 
theta2 = SliceVariate([0.0, 0.0 
width = [1.0, 1.0, 2.0] 
for i in 1:n 
Slice!(thetal, width, logf, :multivar) 
Slice!(theta2, width, logf, :univar) 


siml[i, :, 1] = [thetal[1:2]; exp(thetal[3])] 
Sim2[i, :, 1] = [theta2[1:2]; exp(theta2[3]) ] 
end 


describe(siml) 
describe(sim2) 


SliceVariate Type 


Declaration 


typealias SliceVariate SamplerVariate{SliceTune} 


84 


Chapter 2. Contents 


Mamba.jl Documentation, Release 0.8.2 


Fields 


e value: :Vector{Float 64} : simulated values. 


e tune: :SliceTune: tuning parameters for the sampling algorithm. 


Constructors 


SliceVariate (x::AbstractVector[T<:Real]) 
SliceVariate (x::AbstractVector[T<:Real], tune: :SliceTune) 
Construct a S1iceVariate object that stores simulated values and tuning parameters for slice sampling. 


Arguments 

ex : simulated values. 

*tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a SliceVariate type object with fields set to the values supplied to arguments x and 
tune. 


SliceTune Type 


Declaration 


type SliceTune «: SamplerTune 


Fields 


e width: :Union(Real, Vector}: initial widths defining hyperrectangles from which to simulate values. 


2.4.14 Slice Simplex (SliceSimplex) 


Implementation of the slice simplex sampler as described by Cowles et al. //7/ for simulating autocorrelated draws 
of parameters on the simplex (01,...,04 : 0; > 0, yei 0; = 1} and from a distribution that can be specified up to a 
constant of proportionality. 


Model-Based Constructor 


SliceSimplex (params::ElementOrVector[Symbol]; scale::Real=1.0) 
Construct a Sampler object for which slice simplex sampling is to be applied separately to each of the supplied 
parameters. Parameters are assumed to be continuous and constrained to a simplex. 


Arguments 
*params : stochastic node(s) to be updated with the sampler. 


escale: value0 < scale <= 1 by which to scale the standard simplex to define an initial space from 
which to simulate values. 


Value 


Returns a Sampler{SliceSimplexTune} type object. 
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Example 


See the Asthma, Eyes, and other Examples. 


Stand-Alone Function 


slicesimplex! (v::SliceSimplexVariate, logf:: Function; scale::Real- 1.0) 


Simulate one draw from a target distribution using a slice simplex sampler. Parameters are assumed to be 
continuous and constrained to a simplex. 


Arguments 
ev : current state of parameters to be simulated. 


escale: value 0 < scale <= 1 by which to scale the standard simplex to define an initial space from 


which to simulate values. 


*logf : function that takes a single DenseVector argument of parameter values at which to compute 


the log-transformed density (up to a normalizing constant). 


Value 


Returns v updated with simulated values and associated tuning parameters. 


Example 


KHAAA 
## Multinomial Model 

## y ~ Multinomial(n, rho) 

## rho + Dirdehlet (1; ..., 1) 
KARAHA HHH 


using Mamba 


## Data 

n, k = 100, 5 

rho0 = rand(Dirichlet (ones (k))) 
y = rand(Multinomial(n, rho0)) 


## Log-transformed Posterior(rho) + Constant 


logf = function(rho::DenseVector) 
logpdf (Multinomial (n, rho), y) 
end 


## MCMC Simulation with Slice Simplex Sampling 
t = 10000 
sim = Chains(t, k, names = map(i -> "rho[Si]", 1:k)) 
rho = SliceSimplexVariate(fill(1 / k, k)) 
fof i in l:t 
Slicesimplex! (rho, logf) 
sim[i, :, 1] = rho 
end 
describe (sim) 


p = plot(sim) 
draw(p, filename - "slicesimplexplot") 
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SliceSimplexVariate Type 


Declaration 


typealias SliceSimplexVariate SamplerVariate{SliceSimplexTune} 


Fields 


e value::VectoríFloat64] : simulated values. 


e tune: :SliceSimplexTune : tuning parameters for the sampling algorithm. 


Constructors 


SliceSimplexVariate (x::AbstractVector{T<:Real}) 

SliceSimplexVariate (x::AbstractVector{T<:Real}, tune::SliceSimplexTune) 
Construct a SliceSimplexVariate object that stores simulated values and tuning parameters for slice 
simplex sampling. 


Arguments 

*x : simulated values. 

*tune : tuning parameters for the sampling algorithm. If not supplied, parameters are set to their defaults. 
Value 


Returns a SliceSimplexVariate type object with fields set to the values supplied to arguments 
x and tune. 


SliceSimplexTune Type 


Declaration 


type SliceSimplexTune «: SamplerTune 


Fields 


e scale::Float64: value 0 < scale <= 1 by which to scale the standard simplex to define an initial 
space from which to simulate values. 


The following table summarizes the (d-dimensional) sample spaces over which each method simulates draws, whether 
draws are generated univariately or multivariately, and whether transformations are applied to map parameters to the 
sample spaces. 
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Table 2.3: Summary of sampling methods and their characteristics. 


Model-Based Constructors Stand-Alone Functions 
Method Sample Space | Univariate | Multivariate | Transforma- Univariate | Multivari- 
tions ate 
ABC R? No Yes Yes No No 
AMM R4 No Yes Yes No Yes 
AMWG R4 Yes No Yes Yes No 
BHMC 10,134 No Yes No No Yes 
BMC3 10,134 Yes Yes No Yes Yes 
BMG 10,137 Yes Yes No Yes Yes 
DGS Finite $ c Z4 Yes No No No Yes 
HMC Ri No Yes Yes No Yes 
MALA R4 No Yes Yes No Yes 
MISS Parameter- Yes Yes No No No 
defined 
NUTS R? No Yes Yes No Yes 
RWM RI No Yes Yes No Yes 
Slice SCR? Yes Yes Optional Yes Yes 
SliceSim- d-simplex No Yes No No Yes 
plex 


2.5 Examples 


2.5.1 OpenBUGS 


The following examples are taken from OpenBUGS /4//, and were used in the development and testing of Mamba. 
They are provide to illustrate model specification and fitting with the package, and how its syntax compares to other 
Bayesian modelling software. 


Volume | 


Rats: A Normal Hierarchical Model 


An example from OpenBUGS /4// and section 6 of Gelfand et al. [27] concerning 30 rats whose weights were 
measured at each of five consecutive weeks. 


Model Weights are modeled as 
yi,j ~ Normal (a; + Bilz; — T), 0.) 1—1,:.:4930:3 = Lic. 60 
o ~ Normal (Ha, Ca) 
Bi ~ Normal (ug, og) 
Ha, ug ~ Normal (0, 1000) 
c2 03, a? ~ InverseGamma(0.001, 0.001), 


[o] 


where y; j is repeated weight measurement j on rat 7, and x; is the day on which the measurement was taken. 
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Analysis Program 


using Mamba 


## Data 


rats = Dict{Symbol, 


ry => 
[151, 


134, 
160; 
143, 


171, 


157; 


146, 
i517, 


169, 
157; 
137, 
153, 
¿x => [8 


:xbar 
rats[:N] = 


199; 
199, 
214, 
200, 
198, 
210, 
189, 
201, 
236; 
182, 
208, 
188, 
200, 
223, 
216; 
207, 
187, 
203, 
212, 
203, 
205; 
190, 
191; 
211, 
185, 
207, 
216, 
205; 
180, 
200, 
«0; 


] = 


246, 
249, 
263; 
237; 
230; 
252; 
231, 
248, 
285, 
220, 
261, 
220, 
244, 
270, 
242, 
248, 
234, 
243, 
2589, 
246, 
293; 
225, 
229; 
250, 
2317; 
254. 
261, 
248, 
219; 
244, 
154.0, 


Anyj( 


283, 
293, 
312, 
272; 
280, 
298; 
215; 
297, 
350, 
260; 
313; 
273, 
289, 
326, 
281, 
288, 
280, 
283, 
307, 
286, 
298; 
267, 
272; 
285, 
286, 
303, 
295; 
289; 
298, 
286, 
22410, 


320, 
354, 
328, 
297; 
323, 
S31; 
305, 
338, 
376, 
296; 
352; 
314, 
325, 
358, 
312; 
324, 
316, 
3L 
336, 
321, 
334, 
302, 
302, 
323, 
331, 
345, 
333; 
316, 
291, 
324], 
29.0, . 36.0] 


mean(rats[:x]) 


size(rats[:y], 


1) 


rats[:T] = size(rats[:y], 2) 


rats[:rat] = Int[div(i - 1, 5) + 1 for i in 1:150] 


rats[:week] = Int[(i- 1) $ 5 + 1 for i in 1:150] 
rats[:X] = rats[:x] [rats[:week]] 
rats[:Xm] = rats[:X] - rats[:xbar] 


## Model Specification 
model - Model( 


y = Stochastic(1l, 
(alpha, beta, rat, Xm, s2 c) -> 
begin 
mu = alpha[rat] + beta[rat] 
MvNormal (mu, sqrt(s2 c)) 
end, 
false 


xm 


2.5. Examples 


89 


Mamba.jl Documentation, Release 0.8.2 


) LA 


Stochastic(l, 
s2_alpha) 


alpha = 
(mu_alpha, 
false 

) YA 


-» Normal (mu alpha, 


alpha0 = 
(mu_alpha, 
), 


Logical( 


xbar, mu beta) 


Stochastic ( 
1000), 


mu_alpha = 
() -> Normal (0.0, 
false 

) , 


S2 alpha = Stochastic ( 
() -> InverseGamma(0.001, 0.001), 
false 


dy 


beta = 
(mu_beta, 
false 


Stochastic(l, 


s2_beta) -> Normal (mu_beta, 


Stochastic ( 
1000) 


mu_beta = 
Normal (0.0, 


s2_beta = Stochastic( 


InverseGamma (0.001, 0.001), 


Stochastic ( 


() -> InverseGamma(0.001, 0.001) 


## Initial Values 
inits = [ 
Dict(:y => rats[:yl, 
:mu alpha => 150, 
1:82 beta => 1), 
Dict(:y => rats[:yl, 
:mu alpha => 15, 
:s2 beta => 10) 


:alpha -» fi11(250, 30), 


:mu, beta => 10, 


:alpha -» fi11(20, 30), 


:mu beta => 1, 


## Sampling Scheme 


-> mu alpha - xbar x 


«sde => 1, 


sqrt(s2 alpha)), 


mu beta 


sqrt (s2_beta)), 


:beta => fill(6, 


:beta => fil1(0.6, 
«sz c => 10, 


30), 


:s2_alpha => 1, 


30), 


:s2_alpha => 10, 


scheme = [Slice(:s2_c, 10.0), 
AMWG(:alpha, 100.0), 
Slice([:mu_alpha, :s2 alpha], [100.0, 10.0], :univar), 
AMWG (:beta, 1.0), 
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Slice([:mu beta, :s2 beta], 1.0, :univar)] 
setsamplers! (model, scheme) 
## MCMC Simulations 
sim = mcmc(model, rats, inits, 10000, burnin-2500, thin=2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:10000 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 3750 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 

s2c 37.2543133 6.026634572 0.0695895819 0.2337982327 664.4576 

mu beta 6.1830663 0.108042927 0.0012475723 0.0017921615 3634.4474 


alpha0 106.6259925 


Quantiles: 
2438 25.0% 50.0% 
s2 c 27.778388 33.0906026 36.4630047 
mu beta 5.969850 64 1110307 6.1836454 


alpha0 99.815707 


3.459210115 0.0399435178 0.0526804390 3750.0000 


75.0% 97.3% 
40.5538472 51.9713716 
6.2538831 6.3964953 


104.3369878 106.6105679 108.9124224 113.5045347 


Pumps: Gamma-Poisson Hierarchical Model 


An example from OpenBUGS /41 ] and George et al. [34] concerning the number of failures of 10 power plant pumps. 


Model Pump failure are modelled as 


yi ~ Poisson(0; 

0; ~ Gamma(a, 1/8) 
( 
( 


a ~ Gamma 


i) 


t 
1,1 


) 


B ~ Gamma(0.1, 1), 


where y; is the number of times that pump 7 failed, and t; is the operation time of the pump (in 1000s of hours). 


Analysis Program 


using Mamba 


## Data 
pumps = Dict{Symbol, Any} ( 
Sy => [Sr 1, 5, 14, 3, 19, L, 1, 4, 
st. => [94.3, 15.7. 62.9, 1206, 5.24, 
) 
pumps[:N] = length(pumps[:yl) 


## Model Specification 
model Model ( 
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y = Stochastic(1l, 
(theta, t, N) -> 
UnivariateDistribution[ 
begin 
lambda = theta[i] + t[i] 
Poisson(lambda) 
end 
for i in 1:N 
]s 
false 
), 


theta = Stochastic(1, 
(alpha, beta) -» Gamma(alpha, 1 / beta), 
true 

) , 


alpha = Stochastic ( 
() -> Exponential (1.0) 


beta = Stochastic ( 
() -> Gamma(0.1, 1.0) 


## Initial Values 
inits = [ 
Dict (:y => pumps[:y], :alpha => 1.0, :beta => 1.0, 
:theta => rand(Gamma(1.0, 1.0), pumps[:N])), 
Dict(:y => pumps[:y], :alpha => 10.0, :beta => 10.0, 
:theta => rand(Gamma(10.0, 10.0), pumps[:N])) 


## Sampling Scheme 

scheme = [Slice([:alpha, :beta], 1.0, :univar), 
Slice(:theta, 1.0, :univar)] 

setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc (model, pumps, inits, 10000, burnin-2500, thin=2, chains=2) 
describe (sim) 


## Posterior Predictive Distribution 
ped = predict (sim, :y) 
describe (ppd) 


Results 


## MCMC Simulations 


Iterations = 2502:10000 


92 Chapter 2. Contents 


Mamba.jl Documentation, Release 0.8.2 


Thinning interval = 2 
Chains = 1,2 
Samples per chain = 3750 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
beta 0.93036099 0.517274134 0.00597296721 0.01824153419 804.11618 
alpha 0.69679849 0.264420102 0.00305326034 0.00722593007 1339.06428 
theta[1] 0.05991674 0.025050913 0.00028926303 0.00032725274 3750.00000 
theta[2] 0.10125873 0.078887354 0.00091091270 0.00129985769 3683.18182 
theta[3] 0.08910382 0.037307731 0.00043079257 0.00045660986 3750.00000 
theta[4] 0.11529009 0.030274265 0.00034957710 0.00032583333 3750.00000 
theta[5] 0.59971611 0.316811730 0.00365822675 0.00585119652 2931.65686 
theta[6] 0.60967188 0.134761371 0.00155609028 0.00174949908 3750.00000 
theta[7] 0.86767451 0.669943846 0.00773584520 0.02858200254 549.40360 
theta[8] 0.85445727 0.668132036 0.00771492422 0.02485547216 722.57105 
theta[9] 1.55721556 0.753564449 0.00870141275 0.03109274798 587.38464 
theta[10] 1.98475207 0.405438843 0.00468160451 0.00912748779 1973.09587 
Quantiles 
25$ 25.0% 50.0% 75.0% 97.5% 
beta 0.189558591 0.55034346 0.84013720 1.22683690 2.134324768 
alpha 0.286434561 0.50214938 0.66295652 0.85427430 1.319308645 
theta[1] 0.021291808 0.04164372 0.05696335 0.07435302 0.118041776 
theta[2] 0.008789962 0.04279772 0.08116810 0.13771833 0.305675244 
theta[3] 0.032503106 0.06202730 0.08293498 0.11024076 0.176883533 
theta[4] 0.063587574 0.09389172 0.11222268 0.13443866 0.182555751 
theta[5] 0.153474125 0.36947945 0.54009121 0.77234190 1.364481338 
theta[6] 0.371096082 0.51395467 0.60089332 0.69531957 0.898866079 
theta[7] 0.077416146 0.37391993 0.71230582 1.17659703 2.616100803 
theta[8] 0.072479432 0.35973235 0.69319639 1.16347299 2.655857786 
theta[9] 0.463821785 1.01169918 1.43745818 1.96871605 3.351798915 
theta[10] 1.269842527 1.70167020 1.95748757 2.24000075 2.861197982 
## Posterior Predictive Distribution 
Iterations = 2502:10000 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 3750 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
yil 5.5784000 3.3457654 0038633571 0.042295418 3750.0000 
y[2 1.6009333 1.7532320 0.020244579 0.022399464 3750.0000 
y [3 5.5854667 3.2976980 0.038078537 0.040503717 3750.0000 
y[4] 14.5666667 5.4547523 0.062986054 0.061718435 3750.0000 
y[5 3.1118667 2.4165260 0.027903638 0.035965796 3750.0000 
y[6] 19.0780000 6.0320229 0.069651801 0.071700830 3750.0000 
y[7 0.9008000 1.1756177 0.013574864 0.031805581 1366.2355 
y [8 0.8922667 1.1550634 0.013337523 0.025610086 2034.1808 
y[9 3.2585333 2.4039269 0.027758156 0.069046712 1212.1504 
y[10] 20.7933333 6.2068800 0.071670876 0.113975236 2965.6896 
Quantiles: 
2.5% 25.09 50.03 75,0% 97.5% 
y[1] 1 3 5 8 13 
y[2] 0 0 1 2 6 
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yl3 1 3 5 7 va 
y[4 5 11 14 18 27 
yl5 0 1 3 4 9 
yl6 9 15 19 23 32 
yl7 0 0 1 " 4 
yl8 0 0 i 1 4 
yl? 0 1 3 5 9 
y[10 10 16 20 25 34 


Dogs: Loglinear Model for Binary Data 


An example from OpenBUGS /4//, Lindley and Smith /5//, and Kalbfleisch /49/ concerning the Solomon-Wynne 
experiment on dogs. In the experiment, 30 dogs were subjected to 25 trials. On each trial, a barrier was raised, and an 
electric shock was administered 10 seconds later if the dog did not jump the barrier. 


Model Failures to jump the barriers in time are modelled as 


yi,j = Bernoulli (m; ;) 12 1,,:,5,890;j = 2,...,25 
log(ri;) = azi j-1 + BY — 1— 245-1) 
a, B ~ Flat(—oo, 0.00001) 


where y; ¿ = 1 if dog 1 fails to jump the barrier before the shock on trial j, and 0 otherwise; c; j—1 is the number of 
successful jumps prior to trial 7; and 7; j is the probability of failure. 


Analysis Program 


using Mamba 


## Data 
dogs = Dict{Symbol, Any} ( 
¿Y => 

(OO: 0 10 1 i 1L. Ll 1 1 eo Ngai | 1 1 

0.0 0 Q 0 0 01 0 0 9 0 Q 0.0 1.1 | 1 1 
00000 0 1 1001 L0 1 0 | 1 1 
01100 L X dX qe ou 0 2 31 Lu il: 

0.9 Q-0- 0 0 0 Q 1.4 1 Hh dq» all a | 1 1 
00000011110010111 11111 
6.00002 00 0.00 02241 72 111111 
0 0.0 0 0001 $0 0 LL TA Lu 3 1 
0 0 Q Q O02 012 OQ $t 1.0 i 0 0 0 1 | 19 1 0 
0-0-0-0 29.0.2 X Q- 1 0 1111 NS yal. 

0 00 00 0 0 0 0 0 1 1 T-T. E du» 1 
000001 11100 la, d d E "le XE 
000110 001 1 1 | 111 A Ae T 

0 0 Q O 10 10 1.1 L1 11 bd ql 
00010110111 LT A LU X 

0. 0 0 0 0 0-0 1 T 11 l TALA al. aks di 

0.1 001 0001-01 1 1 1-0 a Su [23 T 

0 00 O.LO 107111101 143 L 1 1 

0 1 0 Q9 0:0 0001 LT 3 1. Ud 1 

0 0-0 0 11 Q 1 0.2 1 0 1 0 12111 |l 111 

0 00 211711 1011 | d1 1171321103341 
00-10 x Q- Li Lcx T LL dL 1l 99 t 1 11 

0 0 0-Q O0. 0 0 1 11111111411111 11 

0 9 OO OO 9 Q.l x 1.0 1-0 0 OQ L L-O' a I d 
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040 0-0 o OQ l 2-0 L I 101.011 111 ot ot 
9 0 10 1 TXTllgiduzolrtisisq121211172112273311 
0.0 Qj 0.L.0 1-1 LI X l.l Li X.l1 X17 Lii 
0 0 0 1.09-L-0 T 1.1.0 1 lox 1 1.1 LX dd at o£ 
0 9 0 Oo LX O12 101 0 1.001.011 11.1 
0 0.0.0 1 L1 1 1 T 32 0 Ll Q L 1 21.21 7 S.1 I 
) 
dogs[:Dogs] = size(dogs[:Y], 1) 
dogs[:Trials] = size(dogs[:Y], 2) 
dogs[:xa] = mapslices(cumsum, dogs[:Y], 2) 
dogs[:xs] = mapslices(x -» collect(1:25) - x, dogs 
dogs[:y] = 1 - dogs[:Y] [:, 2:25] 
## Model Specification 
model = Model ( 
y = Stochastic(2, 
(Dogs, Trials, alpha, xa, beta, xs) -> 
UnivariateDistribution[ 
begin 
p = exp(alpha » xali, j] + beta * xs[i, 
Bernoulli (p) 
end 
for s in. I: Dogs; J in 1:Trials-1 
]s 
false 
js 
alpha = Stochastic ( 
() -> Truncated(Flat(), -Inf, -1e-5) 
E 
A = Logical( 
alpha -> exp (alpha) 
E 
beta = Stochastic( 
() => Truncated(Flat(), -Inf, -1e-5) 
Ls 
B = Logical ( 
beta -> exp (beta) 
) 
) 
## Initial Values 
inits = [ 
Dict (:y => dogs[:y], :alpha => -1, :beta => -1), 
Dict (:y => dogs[:y], :alpha => -2, :beta => -2) 


## Sampling Scheme 
scheme [Slice ([:alpha, 


:beta], 1.0)] 


KHERI 
PRRERPRRR PO 


[:xa], 


31) 
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setsamplers! (model, 


## MCMC Simulations 


scheme) 


sim = mcmc(model, dogs, inits, 


describe (sim) 


10000, burnin=2500, 


thin=2, chains=2) 


Results 


Iterations = 2502:1 
Thinning interval = 
Chains = 1,2 

Samples per chain = 


Empirical Posterior 
Mean 

beta -0.0788601 

B 0.9242336 

alpha -0.2441654 

A 0.7835846 


O O Oo oO 


Quantiles: 
2.5% 
beta -0.10326776 - 
B 0.90188545 
alpha -0.29269315 - 
A 0.74625110 


.011812880 
.010903201 
.024064384 
.018821132 


0000 
2 


3750 


Estimates: 
SD 


oOooc 


25.0$ 
0.086633844 
0.917012804 
0.260167645 
0.770922334 


Naive SE 
.00013640339 
.00012589932 
.00027787158 
.00021732772 


50.0% 


-0.07848501 
0.92451592 
-0.24381478 
04779363276 


MCSE 


oooo 


19405 
-0.07046663 
0.93195884 
-0.22784322 
0.79624909 


ESS 


.00018435162 3750.0000 
.00017012187 3750.0000 
.00045249868 2828.2310 
.00035466850 2816.0882 


97.5$ 
-0.05703080 
0.94456498 
—0,198729T75 
0.81977141 


Seeds: Random Effect Logistic Regression 


An example from OpenBUGS /4//, Crowder [15], and Breslow and Clayton /9/ concerning the proportion of seeds 
that germinated on each of 21 plates arranged according to a 2 by 2 factorial layout by seed and type of root extract. 


Model  Germinations are modelled as 


Tg c Binomial(n;, p;) i= 


logit(p;) = ao + oa; + 09%9 + Q12£1i£2; + bi 
b; ~ Normal (0, 0) 

a0, 41, 03, 012 ~ Normal (0, 1000) 
a° ~ InverseGamma(0.001, 0.001), 


where r; are the number of seeds, out of n;, that germinate on plate ¿; and xı; and x»; are the seed type and root 


extract. 


Analysis Program 


using Mamba 


## Data 
seeds = Dict{Symbol 
sx => [10, 23, 23 
92^ aly 
:n => [39, 62, 81 
30, 51, 7] 


, Any} ( 


p.360, LO 


pl S97 6; 


, 
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seeds[:N] = length(seeds[:r]) 


## Model Specification 
model = Model ( 


r = Stochastic(l1, 
(alpha0, alphal, xl, alpha2, x2, alphal2, b, n, N) -> 
UnivariateDistribution[ 
begin 
p = invlogit (alpha0 + alphal « x1[i] + alpha2 * x2[i] + 
alphal2 » xl[i] + x2[1] + b[i]) 
Binomial(n[il, p) 
end 
for i in 1:N 
ls 
false 
Vy 


b = Stochastic(l, 
s2 -» Normal(0, sqrt(s2)), 
false 

) YA 


alpha0 = Stochastic( 
() -> Normal(0, 1000) 
dy 


alphal = Stochastic ( 
() -> Normal(0, 1000) 
), 


alpha2 = Stochastic ( 
() -> Normal(0, 1000) 
dy 


alphal2 = Stochastic ( 
() => Normal(0, 1000) 


s2 = Stochastic ( 
() -> InverseGamma(0.001, 0.001) 


## Initial Values 
inits = [ 
Dict(:r => seeds[:r], :alpha0 => 0, :alphal => 0, :alpha2 => O, 
:alphal2 => 0, :s2 => 0.01, :b => zeros(seeds[:N])), 
Dict (:r => seeds[:r], :alpha0 => 0, :alphal => 0, :alpha2 => O, 
:alphal2 => 0, :s2 => 1, :b => zeros(seeds[:N])) 
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## Sampling Scheme 


scheme = [AMM([:alpha0, :alphal, :alpha2, :alphal2], 0.01 * eye(4)), 
AMWG(:b, 0.01), 
AMWG(:s2, 0.1)] 
setsamplers! (model, scheme) 
## MCMC Simulations 
sim = mcmc(model, seeds, inits, 12500, burnin=2500, thin=2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:12500 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 5000 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
alpha2 1.310728093 0.26053104 0.0026053104 0.0153996801 286.21707 
alphal 0.088700176 0.26872879 0.0026872879 0.0128300598 438.70341 
alpha0 -0.556154341 0.17595432 0.0017595432 0.0101730837 299.15388 
alphal2 -0.746440855 0.43006756 0.0043006756 0.0251658152 292.04607 
s2 0.085705306 0.09738014 0.0009738014 0.0080848189 145.07755 
Quantiles: 
2.5$ 25.0% 50.0% 75.0% 97.5% 
alpha2 0.8040593795 1.148881827 1.309947687 1.48076318 1.82815608 
alphal -0.4250164771 -0.093637900 0.094390643 0.26007581 0.62353933 
alpha0 -0.9149197759 -0.666632319 -0.551292851 -0.44262420 -0.22244775 
alphal2 -1.5457041398 -1.027576522 -0.757250262 -0.49149187 0.17029702 
s2 0.0011739822 0.021117624 0.059376315 0.11140082 0.35234645 


Surgical: Institutional Ranking 


An example from OpenBUGS /4// concerning mortality rates in 12 hospitals performing cardiac surgery in infants. 


Model Number of deaths are modelled as 


rj ~ Binomial(n;, pi) 
logit(p;) = bi 
b; ~ Normal (u, 0) 
u ~ Normal (0, 1000) 
c? ~ InverseGamma(0.001, 0001), 


where r; is the number of deaths, out of n; operations, at hospital 2. 


Analysis Program 


using Mamba 


## Data 
surgical 


= Dict{Symbol, Any} ( 


98 Chapter 2. Contents 


Mamba.jl Documentation, Release 0.8.2 


:r => [0, 18, 8, 46, 8, 13, 9, 31, 14, 8, 29, 24], 
:n => [47, 148, 119, 810, 211, 196, 148, 215, 207, 
) 
surgical[:N] = length(surgical[:r]) 
## Model Specification 
model = Model ( 
r = Stochastic(l, 
(n, Pr N) a 
UnivariateDistribution[Binomial(n[i]l, plil) 
false 
), 
p = Logical(1, 
b -» invlogit (b) 
), 
b = Stochastic(l, 
(mu, s2) -» Normal(mu, sqrt(s2)), 
false 
), 
mu = Stochastic( 
() -> Normal(0, 1000) 
), 
pop mean - Logical( 


mu -> invlogit (mu) 


Stochastic( 


## Initial Values 


inits = [ 
Dict (:r => surgical[:r], 
Dict (:r => surgical[:r], 


## Sampling Scheme 


scheme = [NUTS(:b), 
Slice([:mu, :s2], 
setsamplers! (model, scheme) 


## MCMC Simulations 
sim mcmc (model, surgical, 
describe(sim) 


-» InverseGamma(0.001, 


0.001) 


97, 256, 360] 


for i in 1:N], 


:b => fill(0.1, surgical[:N]), :s2 => 1, :mu => 0), 
:b => £111(0.5, surgical[:N]), :s2 => 10, :mu => 1 
1.0)] 

inits, 10000, burnin=2500, thin=2, chains=2) 


Results 
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Iterations = 2502:10000 
Thinning interval = 2 
Chains = 1,2 

Samples per chain = 3750 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 


mu -2.550263247 0.151813688 0.001752993476 0.00352027397 1859.8115 
pop mean 0.073062651 0.010097974 0.000116601357 0.00022880854 1947.7088 
s2 0.183080212 0.161182244 0.001861172237 0.00629499754 655.6065 
prl 0.053571675 0.019444542 0.000224526231 0.00059140521 1080.9986 
p[2 0.103203725 0.022015796 0.000254216516 0.00049928289 1944.3544 
p[3 0.071050270 0.017662050 0.000203943787 0.00022426787 3750.0000 
pl4 0.059863573 0.008208359 0.000094781965 0.00033190971 611.6074 
p[5 0.052400628 0.013632842 0.000157418497 0.00052344316 678.3186 
p[6 0.069799258 0.014697820 0.000169715805 0.00026854081 2995.6099 
pr7 0.066927057 0.015703466 0.000181328008 0.00023888938 3750.0000 
p[8 0.122296440 0.023319424 0.000269269516 0.00086456417 727.5137 
pro 0.070386216 0.014318572 0.000165336624 0.00019674962 3750.0000 
p[10 0.077978945 0.019775182 0.000228344129 0.00031544646 3750.0000 
p[11 0.101809685 0.017568938 0.000202868617 0.00037809859 2159.1405 
p[12 0.068534503 0.011579540 0.000133709009 0.00015162331 3750.0000 
Quantiles: 
2.5% 25.0% 50.0% 75.0% 97.5% 

mu -2.878686485 -2.637983754 -2.540103608 -2.455828143 -2.269200544 
pop mean 0.053217279 0.066733498 0.073094153 0.079013392 0.093706084 
s2 0.011890649 0.082427721 0.141031919 0.233947847 0.601137558 
pia 0.017683165 0.039573948 0.052942713 0.067219732 0092219913 
p[2 0.067482073 0.086541186 0.100283541 0.116521584 0.153268132 
p[3 0.040088134 0.059099562 0.070320337 0.080842394 0.110595078 
pl4 0.044965780 0.054145641 0.059405801 0.065149564 0.076754218 
p[5 0.028224773 0.042599948 0.051613569 0.061247876 0.079162530 
p[6 0.043031618  0.059518315 0.069130011 0.079935035 0100122661 
pl7 0.038586653 0.055992241 0.066465865 0.076006492 0.101351899 
p[8 0.082604334 0.105852706 0.121313052 0.137724088 0.171202438 
pro 0.044453383 0.060622475 0.070120723 0.079078759 0.101046045 
p[10 0.044439834 0.064694148 0.075946776 0.089236114 0.122899027 
p[11 0.071966143 0.088360531 0.100133691 0.112632331 0.139968412 
p[12 0.047466661 0.060611827 0.068267225 0.075638053 0.093309250 


Magnesium: Meta-Analysis Prior Sensitivity 


An example from OpenBUGS /41 J. 
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Model Number of events reported for treatment and control subjects in 8 studies is modelled as 


rj ~ Binomial(n5, p; ;) == I: 
pj; ~ Uniform(0, 1) 
ri ~ Binomial (n$, pj ;) 
logit(p; j) = 0i j + logit(p$ ;) 
0i j ~ Normal (j1;, Ti) 
pu ~ Uniform(—10, 10) 


Ti ~ Different Priors, 


where r5 is the number of control group events, out of n5, in study j; ri is the number of treatment group events; and 


i indexes differ prior specifications. 


Analysis Program 


using Mamba 


## Data 
magnesium = Dict{Symbol, Any} ( 
tee oc». Sy. Byori. 1054 3L; 1L, QO, 
nt => [40, 135, 200, 48, 150, 59, 25, 1159], 
£c => [2, 23, T7, 1, 89, 9, 3, 118], 
nc => [36, 135, 200, 46, 148, 56, 23, 1157] 


magnesium[:rtx hcat([magnesium[:rt] for i in 1:6]...)' 
magnesium[:rcx hcat([magnesium[:rc] for i in 1:6]...)' 
magnesium[:s2] = 1 ./ (magnesium[:rt] + 0.5) + 
1 ./ (magnesium[:nt] - magnesium[:rt] + 0.5) + 
1 ./ (magnesium[:rc] + 0.5) + 
1 ./ (magnesium[:nc] - magnesium[:rc] + 0.5) 
magnesium[:s2 0] = 1 / mean(1 ./ magnesium[:s2]) 
## Model Specification 
model = Model ( 
rex = Stochastic(2, 
(nc, pc) -> 
UnivariateDistribution[Binomial(nc[j], peli; j]) for i in 1:6, j in 1:8], 
false 
) , 
pc = Stochastic(2, 
() -> Uniform(0, 1), 


false 
), 


rtx = Stochastic(2, 
(nt, pc, theta) -> 
UnivariateDistribution[ 

begin 
phi = logit (pc[i, jl) 
pt = invlogit(theta[i, j] + phi) 
Binomial (nt[j], pt) 

end 
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for i in 1:6, J in 1:8 
l, 
false 
dy 


theta = Stochastic(2, 
(mu, tau) -> 


UnivariateDistribution[Normal (mu[i], tau[i]) for i in 1:6, j in 1:8], 
false 
) L4 
mu = Stochastic(l, 
() => Uniform(-10, 10), 
false 


dy 


OR = Logical (1, 
mu -> exp (mu) 


da 


tau = Logical(1, 
(priors, s2 0) -> 
Float64[ 

sqrt (priors[1 
sqrt (priors [2 
priors[3], 
sqrt(s2_0 x* (1 / priors[4] - 1) 
sqrt(s2 0) « (1 / priors[5] = 1), 
sqrt (priors[6]) ] 


~ os 


), 


priors = Stochastic(l, 
s2_0 -> 

UnivariateDistribution[ 
InverseGamma (0.001, 0.001), 
Uniform(0, 50), 
Uniform(0, 50), 
Uniform(0, 1), 
Uniform(0, 1), 
Truncated(Normal(0, sqrt(s2 0 / erf(0.75))), 0, Inf) 

] r 


false 


## Initial Values 
inits = [ 
Dict(:rcx -» magnesium[:rcx], :rtx -» magnesium[:rtx], 
:theta => zeros(6, 8), :mu => fill(-0.5, 6), 
Spe => frl1(0.5, 6, 8), ipriofs => [1, 1, 1, 0:5, 0.5, 11), 
Dict(:rcx -» magnesium[:rcx], :rtx -» magnesium[:rtx], 
:theta => zeros(6, 8), :mu => fill(0.5, 6), 
spe => fil1(0.5, 6, 8), :priors => [1, 1, 1, 0.5, 0.5, 1]) 
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## Sampling Scheme 
scheme = [AMWG(:theta, 0.1), 

AMWG(:mu, 0.1), 

Slice(:pc, 0.25, :univar), 

Slace(:praors, 11.0, 5.0, 5:0; 0:25, 0.25, 54.0], sunivar).] 
setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc(model, magnesium, inits, 12500, burnin=2500, thin=2, chains=2) 
describe (sim) 


Results 
Iterations = 2502:12500 
Thinning interval = 2 


Chains = 1,2 
Samples per chain = 5000 


Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
tau[1] 0.55098858 0.35814901 0.0035814901 0.0221132365 262.31486 
tau[2] 1.11557619 0.58886505 0.0058886505 0.0237788755 613.26606 
tau[3] 0.83211110 0.49113676 0.0049113676 0.0222839957 485.75664 
tau[4] 0.47864203 0.26258828 0.0026258828 0.0135868530 373.51920 
tau[5] 0.48624861 0.35359386 0.0035359386 0.0215005369 270.46485 
tau[6] 0.56841884 0.18877962 0.0018877962 0.0058505056 1041.17429 
OR[1] 0.47784058 0.15389133 0.0015389133 0.0066922017 528.79852 
OR[2] 0.42895913 0.32240192 0.0032240192 0.0081170895 1577.59150 
OR[3] 0.43118350 0.18264467 0.0018264467 0.0064385836 804.69879 
OR[4] 0.47587697 0.13947735 0.0013947735 0.0064893426 461.96170 
OR[5] 0.48545299 0.14603013 0.0014603013 0.0083912319 302.85415 
OR[6] 0.44554385 0.14121352 0.0014121352 0.0053818401 688.47941 
Quantiles: 

2.5% 25.0% 50.0% 15:08 97.5% 

tau[1] 0.050143630 0.28821905 0.49325476 0.73991793 1.42033542 
tau[2] 0.326282292 0.70071249 0.98873505 1.39092470 2.65606546 
tau[3] 0.136936046 0.49195419 0.74461372 1.06769203 2.03061696 
tau[4] 0.091858771 0.28921566 0.44085251 0.61784984 1.10639558 
tau[5] 0.028866916 0.23628679 0.42220429 0.65955432 1.37318750 
tau[6] 0.214834142 0.43417486 0.56753402 0.70014948 0.94171060 
OR[1] 0.206501871 0.37431326 0.47128600 0.57044051 0.80062151 
OR[2] 0.107428346 0.27074745 0.38362516 0.52062237 0.99299623 
OR[3] 0.145435475 0.30454141 0.41470065 0.53630024 0.83015778 
OR[4] 0.231777387 0.38069803 0.47049713 0.56112444 0.76292805 
OR[5] 0.207697044 0.38509504 0.48308284 0.59166588 0.75526778 
OR[6] 0.208377218 0.34750042 0.43313882 0.52797553 0.76192141 


Salm: Extra-Poisson Variation in a Dose-Response Study 


An example from OpenBUGS /4// and Breslow /8] concerning mutagenicity assay data on salmonella in three plates 
exposed to six doses of quinoline. 
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Model Number of revertant colonies of salmonella are modelled as 
yi,j ~ Poisson( p; ;) VST, 6 05.387 = PS 
log(mi j) = a + 8log(zj + 10) + ya; + Ai 
a, B, y ~ Normal (0, 1000) 
i,j ~ Normal(0, 0) 
c? ~ InverseGamma(0.001, 0.001), 


where y; is the number of colonies in plate ¿ and dose 7. 


Analysis Program 


using Mamba 


## Data 
salm = Dict{Symbol, Any} ( 
:y => reshape ( 
[15, 21, 29, 16, 18, 21, 16, 206, 33, 27, 41, 60, 339, 38, UL, 20, 2T, 42], 
Si “Oy 
:x => [0, 10, 33, 100, 333, 1000], 
:plate => 3, 
:dose => 6 


## Model Specification 
model - Model( 


y = Stochastic (2, 
(alpha, beta, gamma, x, lambda) -> 
UnivariateDistribution[ 
begin 
mu = exp(alpha + beta x» log(x[j] + 10) + gamma * x[3] + lambda[i, jl) 
Poisson (mu) 
end 
for i in 1:3, j in 1:6 
1s 
false 
Ya 


alpha - Stochastic( 
() -> Normal(0, 1000) 
), 


beta = Stochastic( 
() => Normal(0, 1000) 
), 


gamma = Stochastic ( 
() -> Normal(0, 1000) 
), 


lambda = Stochastic(2, 
S2 -> Normal (0, sqrt(s2)), 
false 

) YA 


S2 = Stochastic( 
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() -> InverseGamma (0.001, 


## Initial Values 
inits = [ 


0.001) 


Dict (:y => salm[:y], :alpha => 0, :beta => 0, :gamma => 0, :s2 => 10, 
:lambda => zeros(3, 6)), 
Dict (:y => salm[:y], :alpha => 1, :beta => 1, :gamma => 0.01, :s2 => 1, 
:lambda => zeros(3, 6)) 
] 
## Sampling Scheme 
scheme = [Slice([:alpha, :beta, :gamma], [1.0, 1.0, 0.1]), 
AMWG([:lambda, :s2], 0.1)] 
setsamplers! (model, scheme) 
## MCMC Simulations 
sim = mcmc(model, salm, inits, 10000, burnin=2500, thin-2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:10000 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 3750 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
s2 0.0690769709 0.04304237136 0.000497010494 0.001985478741 469.961085 
gamma -0.0011250515 0.00034536546 0.000003987937 0.000025419899 184.590851 
beta 0.3543443166 0.07160779229 0.000826855563 0.007122805926 101.069088 
alpha 2.0100584321 0.26156942610 0.003020343571 0.027100522461 93.157673 
Quantiles: 
2.5% 25.0% 50.0% 75.0% 97.5% 
s2 0.0135820292 0.039788136 0.0598307554 0.08740879470 0.17747300708 

gamma -0.0017930379 -0.001347435 -0.0011325733 -0.00090960998 -0.00039271486 
beta 0.2073237562 0.312747679 0.3582979574 0.40006921583 0.48789037325 
alpha 1.5060295212 1.847111545 2.0062727893 2.16556036713 2.50547846044 


Equiv: Bioequivalence in a Cross-Over Trial 


An example from OpenBUGS /4// and Gelfand et al. [27] concerning a two-treatment, cross-over trial with 10 


subjects. 
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Model Treatment responses are modelled as 


yi; Normmal(mi;, 71) i=1,...,10;7 = 1,2 
mij = wt (-1)75-19/2 + (1717 /2 + ô; 
ôi ~ Normal (0, 92) 
I, 0,7 ~ Normal(0, 1000) 


01,02 ~ InverseGamma(0.001, 0.001) 


where y;,; is the response for patient 2 in period 7; and T; j; = 1,2 is the treatment received. 


Analysis Program 


using Mamba 


## Data 
equiv = Dict{Symbol, Any} ( 
:group => [1, 1, 2, 2, 2, 1, 1, 1, 2, 2], 


:y => 
[1.40 1.65 
1.64 1.57 
1.44 1.58 
1.36 1.68 
1.65 15.69 
1:08 1.31 
1.09 1.493 
1.25 1.44 
L.25 1.39 
1230 1:52] 
) 
equiv[:N] = size(equiv[:y], 1) 
equiv[:P] = size(equiv[:y], 2) 
equiv[:T] = [equiv[:group] 3 - equiv[:group] ] 


## Model Specification 
model = Model ( 


y = Stochastic(2, 
(delta, mu, phi, pi, s2 1, T) -> 
begin 
sigma = sqrt (s2_1) 
UnivariateDistribution[ 
begin 
m = mu + (-1)*(T[i, j] - 1) » phi / 2 + (-1)^(j- 1) * pa / 2 + 
delta[i, j] 
Normal (m, sigma) 
end 
for : in 1:10, J din. 1:32 


), 


delta = Stochastic(2, 
s2 2 -» Normal(0, sqrt(s2 2)), 
false 
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mu = Stochastic( 
() -> Normal(0, 1000) 
), 


phi = Stochastic ( 
() -> Normal(0, 1000) 
), 


theta = Logical ( 
phi -> exp (phi) 


pi = Stochastic ( 
() -> Normal(0, 1000) 


s2_1 = Stochastic( 
() -» InverseGamma (0.001, 0.001) 


s2_2 = Stochastic ( 
() -> InverseGamma(0.001, 0.001) 


equiv = Logical ( 
theta -» Int(0.8 < theta < 1.2) 


## Initial Values 
inits = [ 
Dict (:y => equiv[:y], :delta => zeros(10, 2), :mu => 0, :phi => 0, 
¿pi => 0, :s2_1 => 1, :s2_2 => 1), 
Dict (:y => equiv[:y], :delta => zeros(10, 2), :mu => 10, :phi => 10, 
¿pi => 10, :s2_1 => 10, :s2_2 => 10) 


## Sampling Scheme 


scheme = [NUTS(:delta), 
Slice([:mu, :phi, :pi], 1.0), 
Slice([:s2_1, :s2 2], 1.0, :univar)] 


setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc (model, equiv, inits, 12500, burnin-2500, thin=2, chains=2) 
describe (sim) 


Results 
Iterations = 2502:12500 
Thinning interval = 2 


Chains = 1,2 
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Samples per chain - 


Empirical 


s2 2 
s2 1 


Quantiles 


s2 2 
s2 1 


Posterior 
Mean 


40173121833 
.0184397014 
.1874240524 
.0035569545 
.0002921934 
.9751000000 
.4387396416 


2.5% 


.0016375061 
.0017289114 
3979631753 
1723722017 
.8416658455 
. 0000000000 
.3552569594 


5000 


O C» 0 0 0.00 


.014549568 
013837972 
. 086420302 
- 087590520 
.088250458 
1155828169 
.042269208 


0. 
0. 
Da 
-0. 
0. 
Pa 
T. 


Estimates: 


SD 


So C» OQ O OC CO € 


25.0$ 
0056159514 
0074958338 


9395470702 
0000000000 
4110400018 


Naive SE 


.00014549568 
.00013837972 
.00086420302 
.00087590520 
.00088250458 
.00155828169 
.00042269208 


50.0$ 


2432161807 - 
0623573600 - 


.013968228 
.015849718 
.187946319 
.005681830 
.994334281 
.000000000 
.438593809 


MCSE 
.0007329722 
.0005689492 
40032257037 
.0035141650 
0036227671 
- 0036666529 
40013735876 


75.0% 


RPPOOOO 


.024613730 
.025963832 
.130454165 
.053144647 
.054582177 
.000000000 
166525521 


ESS 


.02626 
95873 
.76558 
29903 
.40761 
. 14385 
96847 


OT 98 


.053154674 
.051967161 
.014910965 
.172913654 
.188763456 
.000000000 
.519643109 


Dyes: Variance Components Model 


An example from OpenBUGS /4//, Davies [79], and Box and Tiao /7/ concerning batch-to-batch variation in yields 
from six batches and five samples of dyestuff. 


Model Yields are modelled as 


yi,j ~ Normal (pi, Owithin) 
Ii; ~ Normal (0, Oberween) 
9 ~ Normal (0, 1000) 


G2 thins Cestween ~ InverseGamma(0.001, 0.001), 


where y; j is the response for batch ¿ and sample j. 


Analysis Program 


using Mamba 


## Data 
dyes = Dict{Symbol, Any} ( 
:y => 
[1545, 1440, 1440, 1520, 1580, 
1540, 1555, 1490, 1560, 1495, 
1595, 1550, 1605, 1510; 1560, 
1445, 1440, 1595, 1465, 1545, 
1595, 7630, 1515. 1635, 1625, 
1520, 1455, 1450, 1480, 1445], 
:batches => 6, 
:samples => 5 


dyes[:batch] = 
dyes[:sample] = 


vcat([fill(i, dyes[:samples]) 
vcat (fi11 (collect (1:dyes[:samples]), 


for i in l:dyes[:batches]]...) 
dyes[:batches])...) 
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## Model Specification 


model Model ( 


Stochastic(l, 
(mu, batch, 
false 


yr 


M 


= Stochastic(l, 
(theta, batches, 
dy 


mu 


theta 
O 
), 


Stochastic ( 
-» Normal(0, 1000 


S2 within Stochastic 
() -> InverseGamma (0 


), 


s2 between 
() -> InverseGamma (0 


## Initial Values 
= [ 


inits 


S2 within) 


S2 between) 


-» MvNormal (mu[batch], 


-» Normal(theta, 


) 


( 


001, 0.001) 


Stochastic ( 


«DOT, 9.001) 


Sqrt(s2 within)), 


sqrt (s2_between) ) 


=> 1, :s2_between => 1, 
=> 10, :s2_between => 10, 
] 

thin=2, chains=2) 


Dict(:y => dyes[:y], :theta => 1500, :s2 within 
smu => fil1(1500, dyes[:batches])), 
Dict(:y => dyes[:y], :theta => 3000, :s2 within 
:mu => fil1(3000, dyes[:batches]) ) 
] 
## Sampling Schemes 
scheme = [NUTS([:mu, :theta]), 
Slice([:s2 within, :s2 between], 1000.0) 
scheme2 - [MALA(:theta, 50.0), 
IALA(:mu, 50.0, eye(dyes[:batches])), 
Slice([:s2_within, :s2_between], 1000.0)] 
scheme3 = [HMC(:theta, 10.0, 5), 
HMC (:mu, 10.0, 5, eye(dyes[:batches])), 
Slice([:s2_within, :s2_between], 1000.0)] 
scheme4 = [RWM(:theta, 50.0, proposal=Cosine), 
RWM(:mu, 50.0), 
Slice([:s2_within, :s2_between], 1000.0)] 
## MCMC Simulations 
setsamplers! (model, scheme) 
sim = mcmc (model, dyes, inits, 10000, burnin=2500, 
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describe(sim) 


setsamplers! (model, scheme2) 
sim2 = mcmc(model, dyes, inits, 10000, burnin-2500, thin-2, chains-2) 
describe (sim2) 


setsamplers! (model, scheme3) 
sim3 = mcmc (model, dyes, inits, 10000, burnin-2500, thin=2, chains=2) 
describe (sim3) 


setsamplers! (model, scheme4) 
sim4 = mcmc(model, dyes, inits, 10000, burnin=2500, thin=2, chains=2) 
describe (sim4) 


Results 
Iterations = 2502:10000 
Thinning interval = 2 


Chains = 1,2 
Samples per chain = 3750 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 

s2_between 3192.2614 4456.129102 51.45494673 487.44961486 83.571209 
theta 1526.7186 24.549671 0.28347518 0.37724897 3750.00000 
s2 within 2887.5853 1075.174260 12.41504296 76.89117959 195.52607 
mu[1] 1511.4798 20.819711 0.24040531 0.52158448 1593.30921 
mu[2] 1527.9087 20.344151 0.23491402 0.30199960 3750.00000 
[3] 1552.6742 21.293738 0.24587891 0.70276515  918.08605 
mu[4] 1506.6440 21.349176 0.24651905 0.61821290 1192.57616 
[ 0 1 
[ 0 1 


5] 1578-6636 25.912411 .29459264 .49216105 389.82685 
6] 1487.1934 24.693967 .28514137 .23710390 398.44592 


Quantiles: 
24,55 25.0% 50.0% 75.0% 97.5% 

S2 between 111.92351 815.9012 1651.4605 3269.5663 1.90261752x104 
theta 1475.08243 1513.5687 1527.1763 1540.0550 1.57444106x103 
s2_within 1566.61796 2160.3251 2654.9358 3324.8621 5.65161862x103 
mu[1] 1469.68693 1498.0985 1511.3084 1525.7344 1.55281296x103 
mu[2] 1486.15990 1514.8487 1527.6843 1541.2155 1.56770562x103 
mu[3] 1512.72912 1537.7046 1552.1976 1566.7188 1.59498301x103 
mu[4] 1463.91701 1492.3206 1506.9856 1521.3449 1.54854165x103 
mu[5] 1528.52480 1562.1831 1579.2515 1596.0167 1.62731017x103 
mu[6] 1440.27721 1470.7983 1486.1844 1502.9911 1.54464614x103 


Stacks: Robust Regression 


An example from OpenBUGS /4//, Brownlee //2/, and Birkes and Dodge /4] concerning 21 daily responses of stack 
loss, the amount of ammonia escaping, as a function of air flow, temperature, and acid concentration. 
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Model Losses are modelled as 


yi ~ Laplace(p;, 07) i2 1,...,21 
hi = Bo + pizii + Pazos + B32: 

Bo, P1, G2, B3 ~ Normal (0, 1000) 
c? ~ InverseGamma(0.001, 0.001), 


where y; is the stack loss on day 2; and 21;, 22;, 23; are standardized predictors. 


Analysis Program 


using Mamba 


## Data 
stacks = Dict{Symbol, Any} ( 
Dy => [42, 37, 37, 28, 18, 18, 19, 20, 15, 14, 14, 13, 11,.12,.8, 7; 8, 8, 9; 
15; X5]; 
¿Xx => 
[80 27 89 
80 27 88 
75 25 90 
62 24 87 
62 22 87 
62 23 87 
62 24 93 
62 24 93 
58 23 87 
58 18 80 
58 18 89 
58 17 88 
58 18 82 
5B 19-93 
50 18 89 
50 18 86 
50 19 72 
50: 19 79 
50 20 80 
56-20 82 
TO 20 91] 
) 
Stacks[:N] = size(stacks[:x], 1) 
stacks[:p] = size(stacks[:x], 2) 


stacks[:meanx] = map(j -> mean(stacks[:x][:, jl), 1:stacks[:p]) 
stacks[:sdx] = map(j -> std(stacks[:x][:, jl), l:stacks[:p]) 
stacks[:z] = Float64[ 

(stacks[:x][i, j] - stacks[:meanx][j]l) / stacks[:sdx] [j] 

for i in l1:stacks[:N], j in 1:stacks[:p] 


## Model Specification 
model = Model ( 


y = Stochastic(l1, 
(mu, s2, N) -> 
UnivariateDistribution[Laplace(mu[i], s2) for i in 1:N], 
false 
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) LA 


beta0 = Stochastic( 
() -> Normal(0, 1000), 
false 

); 


beta = Stochastic(l, 
() -> Normal(0, 1000), 
false 

), 


mu = Logical (1, 
(beta0, z, beta) -> betaO + z * beta, 
false 

) , 


s2 = Stochastic ( 
() -> InverseGamma(0.001, 0.001), 
false 

) , 


Sigma = Logical ( 
S2 -» sqrt(2.0) * s2 
), 


b0 = Logical ( 
(beta0, b, meanx) -> beta0 - dot(b, meanx) 
), 


b = Logical(1, 
(beta, sdx) -» beta ./ sdx 
), 


outlier = Logical(l, 
(y, mu, sigma, N) -> 
Float64[abs((y[i] - mu[i]) / sigma) > 2.5 for i in 1:N], 
llr 3, ¿Ap 2] 


## Initial Values 
inits = [ 

Dict (:y => stacks[:y], :beta0 => 10, :beta => [0, 0, 0], :s2 => 10), 
Dict (:y => stacks[:y], :beta0 => 1, :beta => [1, 1, 1], :s2 => 1) 


## Sampling Scheme 

scheme = [NUTS([:beta0, :beta]), 
Slice(:s2, 1.0) ] 

setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc(model, stacks, inits, 10000, burnin-2500, thin=2, chains=2) 
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describe(sim) 


Results 
Iterations = 2502:10000 
Thinning interval = 2 


Chains = 1,2 
Samples per chain = 3750 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 


b[1 0.836863707 0.13085145 0.0015109423 0.0027601754 2247.4171 
b[2 0.744454449 0.33480007 0.0038659382 0.0065756939 2592.3158 
b[3 -0.116648437 0.12214077 0.0014103601 0.0015143922 3750.0000 
b0 -38.776564595 8.81860433 0.1018284717 0.0979006137 3750.0000 
sigma 3.487643717 0.87610847 0.0101164292 0.0279025494 985.8889 
outlier[1 0.042666667 0.20211796 0.0023338572 0.0029490162 3750.0000 
outlier[3 0.054800000 0.22760463 0.0026281519 0.0034398827 3750.0000 
outlier[4 0.298000000 0.45740999 0.0052817156 0.0089200654 2629.5123 
outlier[21 0.606400000 0.48858046 0.0056416412 0.0113877443 1840.7583 
Quantiles: 
2438 25.0% 50.0% 75405 97.5% 
b[1 0.57218621 0.75741345 0.834874964 0.918345319 1.101502854 
b[2 0.16177144 0.52291878 0.714951465 0.933171533 1.476258382 
b[3 -0.36401372 -0.19028697 -0.113463801 -0.036994963 0. 118538277 
b0 -56.70056875 -44.11785905 -38.698338454 -33.409149788 -21.453323631 
sigma 2.17947513 2.86899865 3.348631697 3: 953033535 5.592773118 
outlier[1 0.00000000 0.00000000 0.000000000 0.000000000 1.000000000 
outlier[3 0.00000000 0.00000000 0.000000000 0.000000000 1.000000000 
outlier[4 0.00000000 0.00000000 0.000000000 1.000000000 1.000000000 
outlier[21 0.00000000 0.00000000 1.000000000 1.000000000 1.000000000 


Epilepsy: Repeated Measures on Poisson Counts 


An example from OpenBUGS /4/ ], Thall and Vail /88/ Breslow and Clayton /9/ concerning the effects of treatment, 


baseline seizure counts, and age on follow-up seizure counts at four visits in 59 patients. 


Model Counts are modelled as 


yi,j ~ Poisson(pu;,;) i=1,...,59j=1,...,4 
log(ui,;) = Qo + OBase log (Base, /4) + &raTrt; + agrTrt; log(Base;/4)4- 
Age log(Age;) + ov4V4 + bli + bi; 
bl; ~ Normal(0, 0p1) 
bi į ~ Normal(0, ay) 
a, ~ Normal(0, 100) 


afi, 0% ~ InverseGamma(0.001, 0.001), 


where y;; are the counts on patient 2 at visit j, Trt is a treatment indicator, Base is baseline seizure counts, Age is age 


in years, and V, is an indicator for the fourth visit. 


Analysis Program 
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using Mamba 


## Data 
epil 


Dict{Symbol, Any} ( 


9 


18 


40 20 21 12 


0 
22 


14 13 6 
6 


26 12 
12 


9 12 14 


j 
16 24 10 


13 


9 


37 29 28 29 


12 
18 24 76 29 


13: 35 13 12 


11 14 


8 


9 


22 17 19 16 


18 


11 14 25 15 


10 
19 


10 


102 65 72. 63 


18 11.28 13 
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NO rn c (€) CO 


tBase => 
[11, 11; 6, 8, 606, 27, 12, 52, 23, 10, 52, 33, 18, 42, 87, 50, 18, 111, 18$, 
20, 12, 9, I7, 28, 55, 9, 10, 47, 76, 38, 19, 10, 19, 24, 31, 14, ll, 67, 
Al, 7, 22, 18, 46, 36, 38, 7T, 36, 11, 151, 22, 41, 32, 56, 24, 16, 22, 25, 
135 X2] 

:Age => 
Sly 30) 25, 36, 22, .29,. 91, 42, 37, 29, 30, 24, 239, 36, 26, 207 28, 31, 32; 
21, 29, 21, 32, 25, 30, 40, 19, 22, 18; 32, 20, 30, 18, 24, 30, 35, 27, 20, 
22, 29, 23, 40, 323, 21, 35, 25, 26, 25, 22, 32, 25, 35, 21, 41, 32, 26, 21, 
36, 37], 


:V4 => [0, 0, 0, 1] 

) 

epil[:N] = size(epil[:yl], 1) 

epil[:T] = size(epil[:yl, 2) 

epil[:logBase4] = log(epil[:Base] / 4) 

epil[:BT] = epil[:logBase4] .*« epil[:Trt] 

epil[:logAge] = log(epil[:Age]) 

map (key -> epil[symbol(string(key, "bar"))] = mean(epil[key]), 
:logBase4, :Trt, :BT, :logAge, :V4]) 


## Model Specification 
model = Model ( 


y = Stochastic(2, 
(a0, alpha Base, logBase4, logBase4bar, alpha Trt, Trt, Trtbar, 
alpha BT, BT, BTbar, alpha Age, logAge, logAgebar, alpha V4, V4, 
Vibari, bl, b, N, T) -=> 
UnivariateDistribution[ 


begin 
mu = exp(a0 + alpha Base x* (logBase4[i] - logBase4bar) + 
alpha Trt » (Trt[i] - Trtbar) + alpha BT « (BT[i] - BTbar) + 
alpha Age * (logAge[i] - logAgebar) + 
alpha V4 x (V4[3] - V4bar) + bl[i] + 
bli, JI) 
Poisson (mu) 
end 


for i in 1:N, j in 1:T 
l, 
false 
yr 


bl = Stochastic(l, 
S2 bl -> Normal(0, sqrt(s2_b1)), 
false 

) E 


2.5. Examples 115 


Mamba.jl Documentation, Release 0.8.2 


b = Stochastic (2, 
S2 b -» Normal(0, sqrt(s2 b)), 
false 

) , 


a0 = Stochastic ( 
() -> Normal(0, 100), 
false 

) , 


alpha Base = Stochastic ( 
() -> Normal(0, 100) 
dy 


alpha_Trt = Stochastic ( 
() -> Normal (0, 100) 
), 


alpha BT = Stochastic ( 
() -> Normal(0, 100) 
), 


alpha Age - Stochastic( 
() -> Normal(0, 100) 
), 


alpha_V4 = Stochastic ( 
() -> Normal (0, 100) 
), 


alpha0 = Logical ( 
(a0, alpha Base, logBase4bar, alpha Trt, Trtbar, alpha BT, BTbar, 
alpha Age, logAgebar, alpha V4, V4bar) -> 
a0 - alpha Base * logBase4bar - alpha Trt * Trtbar - alpha BT * BTbar - 
alpha Age * logAgebar - alpha V4 » V4bar 
) YA 


s2 bl = Stochastic ( 
() -> InverseGamma(0.001, 0.001) 
), 


S2 b = Stochastic( 
() -> InverseGamma(0.001, 0.001) 


## Initial Values 
inits = [ 
Dict (:y => epil[:y], :a0 => 0, :alpha Base => 0, :alpha_Trt => 0, 
:alpha BT => 0, :alpha_Age => 0, :alpha V4 => 0, :s2 bl => 1, 
:s2 b => 1, :bl => zeros(epil[:N]), :b => zeros(epil[:N], epil[:T])), 
Dict (:y => epil[:y], :a0 => 1, :alpha Base => 1, :alpha Trt => 1, 
=> 1, :alpha Age => 1, :alpha_V4 => 1, :s2 bl => 10, 
:s2 b => 10, :bl => zeros(epil[:N]), :b => zeros(epil[:N], epil[:T])) 
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## Sampling Scheme 


scheme - [AMWG([:a0, :alpha Base, :alpha Trt, :alpha BT, :alpha Age, 
:alpha V4], 0.1), 
Slice(:b1, 0.5), 
Slice(:b, 0.5), 
Slice([:s2 bl, :8s2 b], 1.0)] 
setsamplers! (model, scheme) 
## MCMC Simulations 
sim = mcmc (model, epil, inits, 15000, burnin-2500, thin=2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:15000 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 6250 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
s2_b 0.13523750 0.031819272 0.00028460022 0.0025401820 156.91007 
s2 bl 0.24911885 0.073166731 0.00065442313 0.0044942066 265.04599 
alpha_V4 -0.09287934 0.083666872 0.00074833925 0.0051087325 268.21356 
alpha Age 0.45830900 0.394536219 0.00352883922 0.0310012419 161.96291 
alpha BT 0.24217000 0.190566444 0.00170447809 0.0163585849 135.70673 
alpha Trt -0.75931393 0.397734236 0.00355744316 0.0337796459 138.63592 
alpha Base 0.91104974 0.135354470 0.00121064718 0.0111438503 147.52807 
alpha0 -1.35617079 1.313240197 0.01174597740 0.1021568814 165.25442 
Quantiles: 
25% 29.0% 50.0% 753.0% 97.5% 
s2_b 0.07155807 0.112591385 0.13626498 0.158032604 0.193715882 
s2 bl 0.13817484 0.197135457 0.23761145 0.289655043 0.422805121 
alpha V4 -0.25504531 -0.148157017 -0.09313603 -0.036681431 0.072099011 
alpha Age -0.19666987 0.176356196 0.41608686 0.696647899 1.305075377 
alpha BT -0.09025014 0.108102968 0.22656174 0.358354280 0.657804969 
alpha Trt -1.63682212 -1.011390894 -0.75653998 -0.480870874 -0.016113397 
alpha Base 0.66318818 0.817700638 0.90268210 0.997417378 1.200619714 
alpha0 -4.16888778 -2.157932918 -1.26343143 -0.436226488 0.866195785 


Blocker: Random Effects Meta-Analysis of Clinical Trials 


An example from OpenBUGS /47 ] and Carlin / 73] concerning a meta-analysis of 22 clinical trials to prevent mortality 
after myocardial infarction. 
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Model Events are modelled as 

~ Binomial(n;, p; ) 1 —1, 222 
tv Binomial(n/, p.) 

logit(p;) = pi 
) = Hi +05 

ui ~ Normal(0, 1000) 

ôi ~ Normal (d, o) 

d ~ Normal(0, 1000) 
c? ~ InverseGamma(0.001, 0.001), 


where rf is the number of control group events, out of nf, in study i; and rt is the number of treatment group events. 


Analysis Program 


using Mamba 


## Data 
blocker = Dict{Symbol, Any} ( 

:rt => 
3, Te 5, 102, 28, 4, 98, 60, 25, 138, 64, 45, 9, 57, 25, 33, 28, 8, 60, 32, 
27, 22]; 
:nt => 
39, UTA, 69; 1533, .355, 59, 945, 632, 278; 1916, S73, 263, 291, 8958, 154, 
207; 251, 151, 174, 209, 391, 680), 
¿rc => 
3, 14. 11, 127, Z7, 6, 152, 48, 37, 188, 52, AT, 16, 45, 31, 38, 12, 6, 3, 
40, 43, 39], 
¿nc => 
39, 116, 93, 1520, 365, 52, 939, 471, 292, 1921, 583, 266, 293, 883, LAR 
213, 122, 154, 134, 218, 364, 674] 


) 
blocker[:N] = length(blocker[:rt]) 


## Model Specification 
model = Model ( 


rc = Stochastic(l, 
qu, ne, N) => 
begin 
pc = invlogit (mu) 
UnivariateDistribution[Binomial(nc[i], pc[i]) for i in 1:N] 
end, 
false 
) , 


rt = Stochastic(1, 
(mu, delta, nt, N) -> 


begin 
pt = invlogit (mu + delta) 
UnivariateDistribution[Binomial(nt[i], pt[i]) for i in 1:N] 
end, 
false 


da 
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mu = Stochastic(l, 
() -> Normal(0, 1000), 
false 


), 


delta = Stochastic(1, 

(d, s2) -» Normal(d, sqrt(s2)), 
false 

) , 


delta new = Stochastic( 
(d, s2) -» Normal(d, sqrt(s2)) 
dy 


d = Stochastic ( 
() -> Normal(0, 1000) 


S2 = Stochastic ( 
() -> InverseGamma(0.001, 0.001) 


## Initial Values 
inits = [ 
Dict(:rc => blocker[:rc], :rt => blocker[:rt], :d => 0, :delta new => 0, 
:s2 => 1, :mu => zeros(blocker[:N]), :delta => zeros(blocker[:N])), 
Dict (:rc => blocker[:rc], :rt => blocker[:rt], :d => 2, :delta new => 2, 
:s2 => 10, :mu => fill(2, blocker[:N]), :delta => fill(2, blocker[:N])) 


## Sampling Scheme 
scheme = [AMWG(:mu, 0.1), 
AMWG([:delta, :delta_new], 0.1), 
Slice ([td, 2821, 1.0) ] 
setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc(model, blocker, inits, 10000, burnin=2500, thin=2, chains=2) 
describe (sim) 


Results 
Iterations = 2502:10000 
Thinning interval = 2 


Chains = 1,2 


Samples per chain 3750 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
S2 0.01822186 0.021121265 0.00024388736 0.0014150714 222.78358 
d -0.25563567 0.061841945 0.00071408927 0.0040205781 236.58613 
delta new -0.25005767 0.150325282 0.00173580684 0.0050219145 896.03592 
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Quantiles: 
2.55 25.0% 50.0% 75.0% 97.5% 
s2 0.0006855452 0.0041648765 0.0107615659 0.024442084 0.07735715 
d -0.3734122953 -0.2959169814 -0.2581848849 -0.218341380 -0.12842580 
delta new -0.5385405488 -0.3279958446 -0.2557849252 -0.177588413 0.07986060 


Oxford: Smooth Fit to Log-Odds Ratios 


An example from OpenBUGS /4/] and Breslow and Clayton /9/ concerning the association between death from 
childhood cancer and maternal exposure to X-rays, for subjects partitioned into 120 age and birth-year strata. 


Model Deaths are modelled as 
r? ~ Binomial(n?,p?) — i— 1,...,120 
ri ~ Binomial(n], p?) 
logit(p?) = Hi 
logit(p;) = pi + log (hx) 
log(v) = a+ fyear; + B2(year? — 22) + b; 


ui ~ Normal(0, 1000) 
b; ~ Normal(0, c) 
a, £1, G2 ~ Normal(0, 1000) 
ow InverseGamma(0.001, 0.001), 


where r? is the number of deaths among unexposed subjects in stratum i, r] is the number among exposed subjects, 
and year; is the stratum-specific birth year (relative to 1954). 


Analysis Program 


using Mamba 


## Data 
oxford 
¿rl => 
[3, 
4, 
Ty 
16; 
4, 


Dict (Symbol, Any} ( 
5; 2; 
12, 8; 
Gp. 53 

17; 


S; 35 
B, 12; 

L3, 8% 
5, 
14, 


5, 
4, 
6, 


11. 
7, 

4, 

6, 


6, 
16; 
8, 4, 
14, 21, 
6, 10, 4, 


6, 
127; 


15; 
17; 
Oy 3, 9; 

IL, 11, 

4, 3, 13, 


4, 
Qi. d 
12, 
6, 
1, 


8, 
11, 14, 
8, 9, 

10, 

7], 


6, 2, 


14, 


35, 
48, 
55, 
61, 


35; 
44, 


43, 
56, 
39, 


49, 
HL, 


53, 
43, 
32; 


40, 
Taz 
64, 


29; 
Ty 
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Tr A. 24 3,5; 4, 10, 2,4 4, 2, I0, 5,; 4, By 4. by 5, 3, 2; Z; 4, Oy åy 1]; 
:nQ => 
[28,.21, 32, 35, 35, 38, 30, 419, 49, 53, 3L, 35, 46, 53, Ol, 40, 29, 44, 52, 
55, 61, 31, 49, 44, 42, 53, 56, 7l, 43, 43, 43, 40, 44, TO, 75, 71, ST, 31, 
42, 46, 47, 55, 63, 91, 43, 39, 35, 32, 53, 49, 75, 64, 69, 64, 49, 29, 40, 
27, 48, 43, 61, 77, 55, 60, 46, 28, 33, 32, 46, 57, 56, 78, 58, 52, 31, 28, 
46, 42, 45, 63, 71, 69, 43, 50, 31, 34, 54, 46, 58, 62, 52, 41, 34, 52, 63, 
59, B8, 62, 47, 53, 5T, Tay 68; 61, 45, 45, 62, 73, 53, 39, 45, 51, 55, 41, 
53, 51, 42, 46, 54, 32], 
:year => 
[-10, 59, --9; =8,.—-8, -B, <7, = Y My YY =O, 65 96, 46, 26, -5, -—5,. -5, 
D, D, CES S, —ü0, Say D, —4 42, 4, 29, 85! HS) By U3525954. 285.29, 2; 
mE. 22. C =Bp Hay Ay HZ, —2y aby 15 elg —L,. Lh Us a +. =i). OF 
OF: O Oy. 0L Or Ou Ui; aya, t 24 Ly ty ay ip dy 2, 2. Spear £y 2) E; 
25 24 2, 3) 37° 35 3; By Sp 3j, 3r 4, 4, 4, Se Ap, 4, 4, 5, 5,5, 5, 5, 5y 6; 
Gr- 6r Gr Op du Ta Tp Tp S By By 9, 9, LOT, 
:K => 120 
) 
oxford[:K] = length(oxford[:r1]) 
## Model Specification 
model = Model ( 
r0 = Stochastic(l, 
(mu, nO, K) -> 
begin 
p = invlogit (mu) 
UnivariateDistribution[Binomial(nO[i], p[i]) for i in 1:K] 
end, 
false 
)y 
rl = Stochastic(l, 
(mu, alpha, betal, beta2, year, b, nl, K) -> 
UnivariateDistribution[ 
begin 
p = invlogit(mu[i] + alpha + betal * year[i] + 
beta2 x (year[i]^2 - 22.0) + b[il) 
Binomial(nl[i], p) 
end 
for i in 1:K 
Iz 
false 
), 
b = Stochastic(l, 
s2 -> Normal (0, sqrt(s2)), 
false 
), 
mu = Stochastic(l, 
() -> Normal (0, 1000), 
false 
), 
alpha = Stochastic ( 


() -> Normal(0, 1000) 
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betal = Stochastic( 


Normal (0, 1000) 


Stochastic( 
Normal(0, 1000) 


Stochastic ( 
-» InverseGamma(0.001, 


## Initial Values 


inits = [ 
Dict (:r0 => oxford[:r0], 
:beta2 => 0, :s2 => 1, 
:mu => zeros (oxford[:K])), 
Dict (:r0 => oxford[:r0], 
:beta2 => 1, :s2 => 10, 
:mu => zeros (oxford[:K])) 


## Sampling Scheme 


:rl => oxford[ 
:b => zeros(oxford[:K]), 


:rl => oxford[ 
:b => zeros(oxford[:K]), 


0.001) 


tETTS 


el], 


:alpha => O0, 


:alpha => 1, 


:betal => 0, 


:betal => 1, 


scheme = [AMWG([:alpha, :betal, :beta2], 1.0), 

Slice(:s2, 1.0), 

Slice(:mu, 1.0), 

Slice(:b, 1.0)] 
setsamplers! (model, scheme) 
## MCMC Simulations 
sim = mcmc(model, oxford, inits, 12500, burnin-2500, thin-2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:12500 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 5000 
Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
beta2 0.005477119 0.0035675748 0.00003567575 0.00033192987 115.519023 
betal -0.043336269 0.0161754258 0.00016175426 0.00133361554 147.112695 
alpha 0.565784774 0.0630050896 0.00063005090 0.00468384860 180.944576 

s2 0.026238992 0.0307989154 0.00030798915 0.00302056007 103.967091 
Quantiles: 
2.5% 25.0% 50.0% 75.0% 97,55 


beta2 -0.0010499046 0.0028489198 
betal -0.0745152363 -0.0543180318 


0.0056500394 
-0.0434425931 


0.0077473623 0.013630865 
-0.0321216097 -0.009920787 
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alpha 0.4438257884 0.5238801187 0.5675039159 0.6051427125 0.695968063 
s2 0.0007134423 0.0033352655 0.0146737037 0.0397132522 0.118202258 


LSAT: ltem Response 


An example from OpenBUGS /4// and Boch and Lieberman /5/ concerning a 5-item multiple choice test (32 possible 
response patters) given to 1000 students. 


Model Item responses are modelled as 


ri j ~ Bernoulli(p; ;) i = 1,..., 1000; j =1,...,5 
logit(pi,j) = 60; — oj 
0; ~ Normal(0, 1) 
a; ^» Normal(0, 100) 
B ~ Flat(0, oo), 


where r; j is an indicator for correct response by student 7 to questions j. 


Analysis Program 


using Mamba 


## Data 
lsat = Dict{Symbol, Any} ( 
:culm => 


[3, 9, 11, 22, 23, 24, 27, 31, 32, 40, 40; 56, 56, 59, 61, 76, 86, 115, 129, 

210, 213, 241, 256, 336, 352, 408, 429, 602, 613, 674, 702, 1000], 
:response => 

[0 0 0 


e 


oO So oo 2 X 


ee 0202 1421 ANA III € (0 0.0010 10 1000 

€» c» pm fr FP oe oo co So FE Fr oe oo G FE. ose 
ove Rie Oo €» 4g qo x» €» qe Pee gegen HES: €»op 1200 
E» pepe Fe or ERIGI oF © BP. IG to [po 


ps pon c o OOo oO S| 
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1 010 

T 011 

1 100 

dt 1 0-1 

1 qo 

11111], 

N => 1000 

) 
lsat[:R] = size(lsat[:response], 1) 
lsat[:T] = size(lsat[:response], 2) 
n = [lsat[:culm] [1]; diff(lsat[:culm]) ] 
idx = mapreduce(i -> fill(i, n[i]), vcat, 1:length(n) ) 
lsat[:r] = lsat[:response] [idx, :] 


## Model Specification 
model = Model ( 


r = Stochastic (2, 
(beta, theta, alpha, N, T) -> 
UnivariateDistribution[ 
begin 
p = invlogit (beta * theta[i] - alpha[j]) 
Bernoulli (p) 
end 
for x in 1:N, J in 1:T 
] r 


false 


theta = Stochastic (1, 
() -> Normal (0, 1), 


alpha = Stochastic(l, 
() -> Normal(0, 100), 


a = Logical(1, 
alpha -> alpha - mean (alpha) 
dy 


beta = Stochastic ( 
() -» Truncated(Flat(), 0, Inf) 


## Initial Values 
inits = [ 
Dict (:r => lsat[:r], :alpha => zeros(lsat[:T]), :beta => 1, 
:theta => zeros(lsat[:N])), 
Dict (:r => lsat[:r], :alpha => ones(lsat[:T]), :beta => 2, 
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:theta => zeros(lsat[:N])) 


## Sampling Scheme 

scheme = [AMWG(:alpha, 0.1), 
Slice(:beta, 1.0), 
Slice(:theta, 0.5)] 

setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc(model, lsat, inits, 10000, burnin-2500, thin=2, chains=2) 
describe (sim) 


Results 
Iterations = 2502:10000 
Thinning interval = 2 


Chains = 1,2 
Samples per chain = 3750 


| 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 


beta 0.80404469 0.072990202 0.00084281825 0.0067491110 116.95963 
a[1] -1.26236241 0.104040037 0.00120135087 0.0025355922 1683.61261 
a[2] 0.48004293 0.069256031 0.00079969977 0.0013701533 2554.91753 
a[3] 1.24206491 0.068338749 0.00078910790 0.0018426724 1375.42777 
a[4] 0.16982672 0.072942222 0.00084226422 0.0012659899 3319.68982 
a[5] -0.62957215 0.086601562 0.00099998871 0.0018787409 2124.79816 
Quantiles: 


2.5$ 25.0$ 50.0$ 75.0% 97.5% 
beta 0.678005795 0.75195190 0.79754709 0.85100547 0.96030934 


a[l] -1.471693755 -1,33040793 -1.25998457 -1.19317801 -1.06168159 
a[2] 0.347262397 0.43161040 0.48023957 0.52718291 0.61527668 
a[3] 1.106413529 1.19669095 1.24105794 1.28858225 1.37451854 
a[4] 0.023253916 0.11970853 0.17099598 0.21998896 0.30858397 
a[5] -0.799988061 -0.68755932 -0.63052599 -0.57168504 -0.46028931 


Bones: Latent Trait Model for Multiple Ordered Categorical Responses 


An example from OpenBUGS /47/ ], Roche et al. [78], and Thissen /89/ concerning skeletal age in 13 boys predicted 
from 34 radiograph indicators of skeletal maturity. 


Model Skeletal ages are modelled as 


0; ^» Normal(0, 100), 


where à; is a discriminability parameter for indicator j, yj, is a threshold parameter, and GO; ;.; is the cumulative 
probability that boy ¿ with skeletal age 0; is assigned a more mature grade than k. 


Analysis Program 
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using Mamba 


## Data 
bones 


Dict{Symbol, Any} ( 


:gamma => reshape ( 


[ 


ZA ZZ zz 
P U 6 oO OO Oo OO WS OW OS 
ZZ4Z2222422242224222424 
ZZ222222222422249 
66 GO 6 o U OO ' OO OS 
ZABZAZABBZBBBSZEO 
st 
œ 
rA 
Z zZzzZzzzZzzZZZZMNOH 0 
686 GO GO BBD DDO TOV O 
Z2Z2Z2ZZ Zo Nros 
O sU pe O 0 
ge US Ps xo 
cd c n n 
OQ r- o C i0 0 (0 XO D p 9 (9 2 O 
pe peo € E E up uy 9 (E ¡EN Ou 
O cO O0 [- CN Kr ss 0o 0 NO O re 
CN 0) 4c) SO. cH o OY. (0) «D oO XO 000 WP cB 
O OY) O «XO CN MH s OY O cJ TN WO 
= | |o ] |o on od 
ZZZ2242242242224224224222242 
66 GO 6 BO WO oO Ow OH OW WA 
ZZ222424242242424242222424 
Z223323 2223323235322 
66 GO C UO o Oo OO OH OW AA 
aa 222222 
Z23232322325323231000 
X XU GO CO DGD HD DBNDAD 
Z2AZAZ 2 2AzZzZ 70070 
ot ON 
m Ps WO n 

AA 
Qa 0) P 1 0) SONAS 
CN c OY OY cq Or GO XO 00 [E St OO c 
s cC oL OO (OY [D M cb CO CP ££) CO St in 
œ 10 CN 10 OO sf OM OO [- (QN OM OS 
C» © €» C © Cp 4) CO P €». (y B) LY 00) 


3.2306, 
543198, 
10.3038], 


2.4970, 
349525, 
71053; 


18799, 
23073, 
551537; 


1.3059, 
0.8021, 
4.0168, 


3.2935, 
2.9495, 
10.4988, 


2.3016, 
2.92689, 
8.2832, 


1.3556, 
148902; 
6.3704, 


0.4927, 
1.5012, 
5,0022, 


4, 
:delta 


34)', 


=> 


0.6324, 
0.978, 

0.9974, 
1.4297, 


3.8376, 
0.6711, 
8.1123; 
1.6766, 


947914, 
0.8246, 
1.0688, 
1.5006, 
1.0954, 


1.0495, 
0.8136, 
0.5381, 
129435, 
2.0906, 


0.7965, 
0.8747, 
1.0331, 
1.368, 

3.4007, 


0.6603, 
0.6968, 
1.6923, 
1.2802, 
3.3085, 


[2.9541, 


0.8272, 
1.1528; 
122656, 
3.385, 


:ncat 


135229]; 


=> 


:grade => reshape ( 


- 
ive) 

= 

[To] 

` 

LO 

- 

LO 

` 

LO 

` - 

mM LO 

ox ` 

e sr [zs] 
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= MAM NO MA 
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Cd CON CN CN CN CN NNN 
— 


2 pepe; 0 2 2 2427 2625 NANG LZ, 2 2 2/2 22 2737353 NaN; Zp NaN; 2; 37547-9707 O 


34, 
:nChild 
:nlInd 


13)', 


=> 13, 
» 34 


## Model Specification 


Stochastic(2, 
delta, 


grade 


=> 


nChild, nind) 


gamma, 


theta, 


(ncat, 


begin 
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p = Array(Float64, 5) 
UnivariateDistribution[ 
begin 
n = ncat [Jj] 
p[1] = 1.0 
for k in 1: (n - 1) 
Q = invlogit (delta[j] * (theta[i] - gamma[j, k])) 
pik] -= Q 
pik + 1] = 0 
end 
Categorical (p[1:n]) 
end 
for i in l:nChild, J in 1:nInd 
] 
end, 
false 
) 
theta = Stochastic(l, 
() -> Normal(0, 100) 
) 
) 
## Initial Values 
inits = [ 
Dict (¿grade => bones[:grade], :theta => [0.5,1,2,3,5,6,7,8,9,12,13,16,18]), 
Dict (:grade => bones[:grade], :theta => [1,2,3,4,5,6,7,8,9,10,11,12,13]) 


## Sampling Scheme 


scheme = [MISS(:grade), 
AMWG(:theta, 0.1) ] 

setsamplers! (model, scheme) 

## MCMC Simulations 

sim = mcmc(model, bones, inits, 10000, burnin-2500, thin=2, chains=2) 

describe (sim) 

Results 

Iterations = 2502:10000 

Thinning interval = 2 

Chains = 1,2 

Samples per chain = 3750 

Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 

theta[l 0.32603385 0.20640874 0.0023834028 0.0039448110 2737.81276 
theta[2 1.37861692 0.25824308 0.0029819342 0.0058663965 1937.82503 
theta[3 2.35227822 0,.,27998526 .0.0032329913 0.0067161153 1737.93707 
theta[4 2.90165730 0.29713320 0.0034309987 0.0078730921 1424.33353 
theta[5 5.54427283 0.50242324 0.0058014839 0.0169090038 882.88350 
theta[6 6.70804782 0.57206890 0.0066056827 0.0221532973 666.83738 
theta[7 6.49138381 0.60154625 0.0069460578 0.0219158412 753.39330 
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theta[8] 
theta[9] 
theta[10] 
theta[11] 
theta[12] 

] 


theta[13 
Quan 


the 
the 
the 
the 
the 
theta 
theta 
the 
theta 
theta[10 
theta[11 
theta[12 
theta[13 


ta 
ta 
ta 
ta 
ta 


DATA 01 4» WN r5 


ta 


Xe) 


tiles: 


C 3: 01 01 € N dq c c 


.93701249 
403595289 
493125529 
.53686992 
.81482824 
.93028146 


2.5% 


41235555 
.91705346 
.78287586 
.32940825 
.59142954 
.56649066 
.38663728 
.47304526 
.80477915 
.64129157 


9.83558611 


. 79290437 
.61843069 


.73636136 
.65172497 
.69360918 
«92271697 
.54261736 
.72458739 


S c O OQ O G 


250405 

.19557824 
.19969433 
.17136780 
.69621746 
.20543314 
» 30983797 
- 07628064 
- 43125608 
- 60559136 
- 48379528 
- 88717498 
.45889470 
.42289725 


errre 
oO U00nrPo0000U0 MN NPpo 


oo0oo0oo0oooO 


.0085027686 
.0075254717 
.0080091090 
.0106546132 
.0062656056 
.0083668145 


50.0$ 

499981559 
«36116575 
739623189 
- 89121336 
. 53392246 
- 70666338 
- 46533033 
. 96072241 
.01498109 
.89611699 
.49029895 
.79840132 
.90719268 


Onn 0 N PO i c» 


o] 


he 
he 


rR 
au 


Q O O CO O 


errre 
ONNO Qo O) -1 O1 CQ. NH. O 


.0336199950 
0233182299 
0282955741 
. 0493587234 
. 0210976666 
0323302069 


75.0% 
.45840506 
9371751273 
.53035766 
10758151 
.86525435 
.09569168 
.88636840 
.45344704 
.46962522 
.37737647 
.15757004 
.15824313 
.41900248 


479. 
781. 
600. 
349. 
661. 
502. 


71875 
15561 
88678 
46912 
48275 
30161 


97,3% 

. 7174563 
.9466119 
.9211580 
.4945343 
.5867245 
8229872 
.7051374 
22856733 
- 3024722 
- 3873043 
.4263451 
,9593310 
.9895761 


Inhalers: Ordered Categorical Data 


An example from OpenBUGS /47 ] and Ezzet and Whitehead /23/ concerning a two-treatment, two-period crossover 
trial comparing salbutamol inhalation devices in 286 asthma patients. 


Model Treatment responses are modelled as 


Ri =j 


if Yit € [aj-1, aj) 


logit (Qi tj) = — (aj + Hs; t + bi) 
fii = 8/2 + T/2 


pa = —B/2 — 7/2- K 
H21 = —B/[2 + 7/2 
pas = B/2-—7/2+k 


b; ~ Normal (0, 0) 

a[1] ~ Flat(—1000, a[2]) 
a[2] ~ Flat(—1000, a[3]) 
a[3] ~ Flat(—1000, 1000) 
B ~ Normal (0, 1000) 
m ~ Normal(0, 1000) 
& ~ Normal (0, 1000) 


i21,..,45£21,2;7 2 1,...,3 


c? ~ InverseGamma(0.001, 0.001), 


where Ri + is a 4-point ordinal rating of the device used by patient 7, and (2; ;,; is the cumulative probability of the 
rating in treatment period t being worse than category j. 


Analysis Program 


128 


Chapter 2. Contents 


Mamba.jl Documentation, Release 0.8.2 


using Mamba 


## Data 
inhalers = Dict{Symbol, Any} ( 
spattern => 
a: TA 2 
12 3 41 
:Ncum => 
59 157 173 175 186 253 270 271 271 278 280 281 282 285 285 286 
122 170 173 175 226 268 270 271 278 280 281 281 284 285 286 286]', 
streat => 
L. -=1 
-1 1], 
:period => 
E =1 
a 
¿Carry => 
0 -1 
0 1], 
:N => 286, 
:T => 2, 
:G => 2, 
:Npattern => 16, 
¿Ncut => 3 


222333 34444 
234123412 3 4]', 


inhalers[:group] = Array(Int, inhalers[:N]) 
inhalers[:response] = Array(Int, inhalers[:N], inhalers[:T]) 


i= 1 
for k in 1:inhalers[:Npattern], g in l:inhalers[:G] 
while i <= inhalers[:Ncum][k, g] 


inhalers [:group] [i] = g 
for t in l:inhalers[:T] 
inhalers[:response][i, t] = inhalers[:pattern][k, t] 
end 
i += 1 
end 


end 


## Model Specification 
model = Model ( 


response = Stochastic (2, 
(al, a2, a3, mu, group, b, N; T) -> 
begin 


a = Float64[al, a2, a3] 
UnivariateDistribution[ 
begin 
eta = mu[group[i], t] + b[il] 
p = ones (4) 
for 3 in 1:3 


Q = invlogit(-(a[j] + eta)) 
p[j] -- Q 
p[j + 1] = O 

end 


Categorical (p) 
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end 
for i in 1:N, t in 1:T 


mu = Logical(2, 
(beta, treat, pi, period, kappa, carry, G, T) -> 
| beta * treat[g, t] / 2 + pi * period[g, t] / 2 + kappa * carryl[g, t] 
for g in 1:G, t in 1:T ], 
false 
), 
b = Stochastic(l, 
s2 -> Normal(0, sqrt(s2)), 
false 
), 
al = Stochastic( 
a2 -» Truncated(Flat(), -1000, a2) 
), 
a2 = Stochastic( 
a3 -» Truncated(Flat(), -1000, a3) 
), 
a3 = Stochastic ( 
() => Truncated(Flat(), -1000, 1000) 
), 
beta = Stochastic( 
() -> Normal(0, 1000) 
), 
pi = Stochastic ( 
() -> Normal(0, 1000) 
), 
kappa = Stochastic( 
() -> Normal(0, 1000) 
), 
S2 = Stochastic( 
() -> InverseGamma(0.001, 0.001) 
) 
) 
## Initial Values 
inits = [ 
Dict (:response => inhalers[:response], :beta => 0, :pi => 0, :kappa => O, 
sal => 2, :a2 => 3, :a3 => 4, :s2 => 1, :b => zeros(inhalers[:N])), 
Dict (:response => inhalers[:response], :beta => 1, :pi => 1, :kappa => O, 
sal => 3, :a2 => 4, :a3 => 5, :s2 => 10, :b => zeros(inhalers[:N])) 
] 
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## Sampling Scheme 
scheme = [AMWG(:b, 0.1), 

Slice([:al, :a2, :a3], 2.0), 

Slice([:beta, :pi, :kappa, :s2], 1.0, :univar)] 
setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc(model, inhalers, inits, 5000, burnin=1000, thin=2, chains=2) 
describe (sim) 


Results 


Mice: Weibull Regression 


An example from OpenBUGS /4/ /, Grieve /40/, and Dellaportas and Smith /20] concerning time to death or censoring 
among four groups of 20 mice each. 


Model Time to events are modelled as 
ti ~ Weibull(r, 1/u7) i—1,...,20 


log(u;) = z; 8 
Br ~ Normal(0, 10) 
r ~ Exponential(1000), 


where t; is the time of death for mouse 7, and z; is a vector of covariates. 


Analysis Program 


using Mamba 


## Data 
mice = Dict{Symbol, Any} ( 
iG => 


[12. 1 21 25 11 26 27 30°13 12 2120 23 25 23 29 35 NaN 31 36 
32 27 23 12 18 NaN NaN 38 29 30 NaN 32 NaN NaN NaN NaN 25 30 37 27 
22 26 NaN 28 19 15 12 35 35 10 22 18 NaN 12 NaN NaN 31 24 37 29 
27 18 22 13 18 29 28 NaN 16 22 26 19 NaN NaN 17 28 26 12 17 26], 


[0 0 000 0 0 000 00 0 0 0 Q O 4000 
00 000 40 40 00 0 40 0 40 40 40 40 0 000 
001000 0 0 000 0024 0 40 400 000 
0 0. 00 0 0 Q 2000 0029 10 0 00 © 0 0] 

) 

mice[:M] = size(mice[:t], 1) 

mice[:N] = size(mice[:t], 2) 


## Model Specification 
model = Model ( 


t = Stochastic(2, 
(r, beta, tcensor, M, N) -> 
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UnivariateDistribution[ 


begin 
lambda = exp(-beta[i] / r) 
O « lambda « Inf ? 
Truncated(Weibull(r, lambda), tcensor[i 
Uniform(0, Inf) 
end 
for i in LM, J in I:N 
l, 
false 
VG 
r = Stochastic ( 
() -» Exponential (1000) 
)s 
beta = Stochastic(l, 
() -> Normal(0, 10), 
false 
), 
median = Logical(1, 


(beta, r) -» exp(-beta / r) * log(2)^(1 / r) 


), 


veh control = Logical( 
beta -> beta[2] - beta[1] 
), 
test sub = Logical( 
beta -> beta[3] - beta[1] 
), 
pos control - Logical( 
beta -> beta[4] - beta[1] 


## Initial Values 


inits = [ 
Dict (:t => mice[:t], :beta => fill(-1, mice[:M]), 
Dict (:t => mice[:t], :beta => fill(-2, mice[:M]), 


## Sampling Scheme 

[MISS (:t), 
Slice(:beta, 1.0, 
Slice(:r, 0.25) ] 
setsamplers! (model, scheme) 


scheme = 
:univar), 


## MCMC Simulations 
sim = mcmc (model, 
describe (sim) 


mice, inits, 20000, burnin=2500, 


, jl, Inf) 

r => 1.0), 

r => 1.0) 
thin=2, chains=2) 
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Results 


Leuk: Cox Regression 


An example from OpenBUGS /47/ ] and Ezzet and Whitehead /25/ concerning survival in 42 leukemia patients treated 
with 6-mercaptopurine or placebo. 


Model Times to death are modelled using the Bayesian Cox proportional hazards model, formulated by Clayton 
[15] as 


dN;(t) ~ Poisson(J;(t)dt) i=1,...,42 
1,(t)dt = Y.(t) exp(8Z;)dAo(t) 
B ~ Normal(0, 1000) 
dAo(t) ~ Gamma(cdA¿(t), 1/c) 
dA (t) = rdt 
c — 0.001 
r — 0.1, 
where d.N;(t) is a counting process increment in time interval |t, t + dt) for patient 7; Y¿(t) is an indicator of whether 


the patient is observed at time t; z; is a vector of covariates; and dAo(t) is the increment in the integrated baseline 
hazard function during |t, t + dt). 


Analysis Program 


using Mamba 


## Data 
leuk = Dict{Symbol, Any} ( 
:t obs => 
1, 1, 2, 2, 3, Ar 4, 5, 5, 8, 8, 8, 8, 11, I1, 12, 12, 15, 17, 22, 23, 6, 
6, 6, 6, 7, 9, 10; 10, 11, 13, 16, 17, 19, 20, 22, 23, 25, 32, 32, 34, 35], 
fail => 
T, Ls By BY de De. 1,4 e ik. 1; 1. 1,4 Ae Le ae de des Ge 1; 1, l1, de 1, 9. 
14054 4. 05a L. X05 4 0 15 $4 0; OF Oy 0 "Ol, 
:Z => 
Oo, 0.5: Uso Deo, 0:5, O35; 0.5, 0.5, 0.5, 015, Ma 0.5) Oud Did Di Sy 
0.5, 0.5, 0.5, 0.5, 0.5, Q5, 0.5, =0.5, —0.5, -0.5, 05.5, -0.5, -—0:5, 
=0..5,. 2045, 0207 OD -O. 5; 20.5, 204,5. OD. —05.5, 30.5, -025, 049, 
-0.5,; -0.5]; 
¡tus [1,.2,.3, 4, 5,4 6, 7,9, 10, I1, 12, 13, 15, TO, 17, 22, 23, 35] 


leuk[:N] = N = length(leuk[:t_obs]) 
leuk[:T] = T = length(leuk[:t]) - 1 


leuk[:Y] = Array(Int, N, T) 

leuk[:dN] = Array(Int, N, T 

for i in 1:N 
for j in lI:T 

leuk[:dN][i, j] = leuk[:fail] [i] » (leuk[:t_obs] [i] == leuk[:t]1[3]) 

leuk[:Y][i, 3] Int (leuk[:t_obs] [i] >= leuk[:t] (JJ) 

end 
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## Model Specification 
model = Model ( 


daN = Stochastic(2, 
(Y, beta, Z, dLO, Ny T) => 
UnivariateDistribution[ 
Y[i, j] > 0 ? Poisson(exp(beta * Z[i]) 
for i in 1:N, j in 1:T 
ie 
false 
ys 


mu = Logical(1i, 


(c, r; t) -> C* r + (t[2:end] - t[1: (end - 1)]), 
false 

), 

dLO = Stochastic(l, 


(mu, c, T) -> UnivariateDistribution[Gamma (mu[j], 
false 


); 


beta = Stochastic ( 
() -> Normal (0, 1000) 


), 


SO = Logical(i, 
dLO -» exp(-cumsum(dL0)), 
false 

), 


S treat = Logical(1, 


(SO, beta) -> S0.^exp(-0.5 » beta) 
), 
S placebo = Logical(1, 
(SO, beta) -> S0.^exp(0.5 * beta) 
) 
) 
## Initial Values 
inits [ 


:beta => 0, :dLO => fill(1, 


Dict(:dN -» leuk[:dN], 
( :beta => 1, :dLO => fill(2, 


:dN => leuk[:dN], 


## Sampling Scheme 


scheme = [AMWG(:dLO, 0.1), 
Slice(:beta, 3.0) ] 
setsamplers! (model, scheme) 


* dL0[j]) 


1/ c) 


leuk[:T 


leuk[:T 


Flat() 


for j in 1:T], 
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## MCMC Simulations 


sim = mcmc (model, leuk, inits, 10000, burnin=2500, thin=2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:10000 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 3750 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
beta 1.552064427 0.424977799 0.00490722093 0.0121343026 1226.59967 
S treat[1] 0.983021837 0.014032858 0.00016203749 0.0006444873 474.09293 
S treat[2] 0.966246426 0.020819923 0.00024040776 0.0009762155 454.84854 
S treat[3] 0.956207131 0.024534430 0.00028329919 0.0011874261 426.91239 
S treat[4] 0.936432950 0.031636973 0.00036531230 0.0015140463 436.62796 
S treat[5] 0.913982241 0.037982886 0.00043858859 0.0017354084 479.04081 
S treat[6] 0.879372323 0.047653044 0.00055024996 0.0021437962 494.09936 
S treat[7] 0.867528817 0.051602796 0.00059585777 0.0024545959 441.96358 
S treat[8] 0.821750962 0.064689997 0.00074697574 0.0030256573 457.12479 
S treat[9] 0.805557685 0.068612901 0.00079227353 0.0032183437 454.51343 
S treat[10] 0.771831536 0.076942474 0.00088845517 0.0035776788 462.51903 
S treat[11] 0.735657019 0.085534730 0.00098766999 0.0040566359 444.58306 
S treat[12] 0.712600209 0.089598298 0.00103459203 0.0040292944 494.47179 
S treat[13] 0.691135357 0.094685927 0.00109333891 0.0044879053 445.12656 
S treat[14] 0.664423491 0.098857036 0.00114150273 0.0046673584 448.61405 
S treat[15] 0.636423003 0.102857125 0.00118769178 0.0047872478 461.63310 
S treat[16] 0.565616003 0.112893079 0.00130357699 0.0049341458 523.49276 
S treat[17] 0.471034334 0.120102602 0.00138682539 0.0050776645 559.47003 
S placebo[1] 0.927789861 0.050170096 0.00057931437 0.0023120443 470.86627 
S placebo[2] 0.859431833 0.067291385 0.00077701399 0.0030529117 485.83685 
S placebo[3] 0.820804570 0.074342778 0.00085843646 0.0034349005 468.43490 
S placebo[4] 0.748344200 0.085488930 0.00098714114 0.0038433955 494.75434 
S placebo[5] 0.671088723 0.090818803 0.00104868521 0.0036921538 605.05098 
S placebo[6] 0.564655613 0.097783399 0.00112910544 0.0038182815 655.83466 
S placebo[7] 0.532173306 0.099628193 0.00115040728 0.0042364979 553.03234 
S placebo[8] 0.418874416 0.097451300 0.00112527068 0.0038432054 642.96611 
S placebo[9] 0.383210551 0.095687796 0.00110490749 0.0036248738 696.83078 
S placebo[10] 0.317122087 0.091201496 0.00105310416 0.0031779306 823.59766 
S placebo[11] 0.256739191 0.086400289 0.00099766461 0.0029358329 866.09938 
S placebo[12] 0.223014153 0.082260904 0.00094986710 0.0026813211 941.21603 
S placebo[13] 0.195547225 0.079430544 0.00091718492 0.0029106845  744.70591 
S placebo[14] 0.164855445 0.074276591 0.00085767220 0.0026970503 758.44802 
S placebo[15] 0.137032754 0.068763778 0.00079401571 0.0026184621  689.64703 
S placebo[16] 0.083796000 0.054748991 0.00063218689 0.0020660044 702.24679 
S placebo[17] 0.040920336 0.037737842 0.00043575906 0.0012818385 866.73735 
Quantiles: 
225% 25.0% 50.0% 75.0% 97.5% 
beta 0.7524244645 1.259092877 1.540715015 1.829373993 2.41862572 
S_treat[1] 0.9462141591 0.977211258 0.986955387 0.992864719 0.99836877 
S_treat[2] 0.9150951603 0.955732194 0.970764331 0.981528195 0.99312428 
S_treat[3] 0.8974993157 0.943194903 0.960671242 0.974341293 0.98958205 
S_treat[4] 0.8599068641 0.919023530 0.941699242 0.959578177 0.98196246 
S_treat[5] 0.8239755135 0.892499135 0.919841738 0.941506275 0.97166964 
S treat [6] 027738755125 0.850772834 0.885177533 0.913944652 0.95521105 
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S treat[7] 0.7531822416 0.836159276 0.873267366 0.904856224 0.95037821 

S treat[8] 0.6801713643 0.780967298 0.828861810 0.868789540 0.92883388 

S treat[9] 0.6555328459 0.762836917 0.813277192 0.854664506 0.91930265 

S treat[10] 0.6026848684 0.723608005 0.779448002 0.826762737 0.90169913 

S treat[11] 0.5503082957 0.681881638 0.743447395 0.797449392 0.87990722 

S treat[12] 0.5233565529 0.654970518 0.720927823 0.777608202 0.86466162 

S treat[13] 0.4922868942 0.631085044 0.699537585 0.758920006 0.85262814 

S treat[14] 0.4597297883 0.598392933 0.670665550 0.735415731 0.83825032 

S treat[15] 0.4269750729 0.566537132 0.641228870 0.709931594 0.82208295 

S treat[16] 0.3438735907 0.487880437 0.566936069 0.644918756 0.77749139 

S treat[17] 0.2466734959 0.383395600 0.469857377 0.555064846 0.70280881 

S placebo[1] 0.7997904171 0.903185030 0.939595572 0.964924422 0.99089079 

S placebo[2] 0.7055958749 0.819313313 0.869352795 0.908385797 0.96318697 

S placebo[3] 0.6563988800 0.773124226 0.828159645 0.876023887 0.94337131 

S placebo[4] 0.5660793194 0.692444549 0.754818343 0.810412223 0.89462480 

S placebo[5] 0.4851605247 0.609481976 0.675431460 0.736195314 0.83615527 

S placebo[6] 0.3696872931 0.497149446 0.566089714 0.632288659 0.74932879 

S placebo[7] 0.3402856402 0.462939149 0.532290556 0.601897323 0.72484128 

S placebo[8] 0.2379614336 0.349405859 0.415075601 0.483568964 0.62050327 

S placebo[9] 0.2091293870 0.314947858 0.378301304 0.447909630 0.58210955 

S placebo[10] 0.1562465305 0.250265102 0.311673305 0.378896633 0.51089391 

S placebo[11] 0.1092657207 0.193262012 0.249637651 0.312589668 0.44360059 

S placebo[12] 0.0845659713 0.162032458 0.216266931 0.276356661 0.40448425 

S placebo[13] 0.0650227582 0.137218626 0.187654643 0.246139543 0.37311156 

S placebo[14] 0.0479431220 0.110429658 0.156032281 0.209723502 0.33168432 

S placebo[15] 0.0342784539 0.086171750 0.127242826 0.176712122 0.29500763 

S placebo[16] 0.0125569906 0.042952635 0.072131767 0.112032446 0.22046612 

S placebo[17] 0.0021678381 0.013959709 0.029822826 0.055822282 0.14078639 
Volume ll 


Birats: A Bivariate Normal Hierarchical Model 


An example from OpenBUGS /4// and section 6 of Gelfand et al. 
measured at each of five consecutive weeks. 


Model Weights are modeled as 
Y, j ~ Normal (pi j, oc) pedes 
Bá = Pri + Bam; 
Bi ~ MvNormal (ug, 2) 
Ug ~ MvNormal (fo) "n e 
200 0 


Y ~ InverseWishart (2 | 


0 " 


02. ~ InverseGamma(0.001, 0.001), 


[27] concerning 30 rats whose weights were 


,30: 2 1,...,5 


where y;,; is repeated weight measurement j on rat ?, and x; is the day on which the measurement was taken. 


Analysis Program 


using Mamba 
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## Data 
birats = Dict{Symbol, Any} ( 

:N => 30, :T => 5, 

ix o (8.0; 15.0, 22.0, 29.0, 36.0], 

:Y => [151 199 246 283 320 
145 199 249 293 354 
147 214 263 312 328 
155. 200 237 272 297 
135 188 230 280 323 
159 210 252 298 331 
141 189 231 275 305 
159 201 248 297 338 
177 236 285 350 376 
134 182 220 260 296 
160 208 261 313 352 
143 188 220 273 314 


154 200 244 289 325 
171 221 270 326 358 
163 216 242 281 312 
160 207 248 288 324 
142 187 234 280 316 
156 203 243 283 317 
l57 212 259 307 336 
152 203 246. 286 321 


154 205 253 298 334 
139 190 225 267 302 
146 191 229 272. 302 
157 211 250 285. 323 
132 185 237 286 331 
160 207 257 303 345 
169 216 261 295 333 
157 205 248 289 316 
137 180 219 258 291 
153 200 244 286 324], 
:mean => [0.0, 0.0], 


:var => [1.0e6 0.0 
0.0 1.0e6], 
:Omega => [200.0 0.0 
0.0 0.2] 


## Model Specification 
model = Model ( 


Y = Stochastic(2, 
(beta, x, sigmaC, N, T) -> 
UnivariateDistribution[ 
Normal(beta[i, 1] + beta[i, 2] * x[j], sigmaC) 
for 3 in 1:N, J in 1:T 
l, 
false 
), 


beta = Stochastic(2, 
(mu_beta, Sigma, N) -> 
MultivariateDistribution[ 
MvNormal (mu beta, Sigma) 
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for i in 1:N 


l; 


false 
dy 
mu beta = Stochastic(l, 
(mean, var) -> MvNormal(mean, var) 


), 


Sigma = Stochastic(2, 
Omega -» InverseWishart (2, Omega), 
false 


), 


sigma2C = Stochastic ( 
() -> InverseGamma(0.001, 0.001), 
false 


), 


SigmaC = Logical ( 
sigma2C -> sqrt (sigma2C) 


## Initial Values 
inits = [ 
Dict (:Y => birats 
:mu beta => 
Dict(:Y => birats 
:mu beta => 


:Y], :beta -» repmat([100 6], birats[:N], 1), 
0, 0], :Sigma => eye(2), :sigma2C => 1.0), 
:Y], :beta => repmat([50 3], birats[:N], 1), 


[ 
[ 
[ 
[10, 10], :Sigma => 0.3 * eye(2), :sigma2C => 10.0) 


## Sampling Scheme 

Scheme = [AMWG([:beta, :mu beta], repmat([10.0, 1.0], birats[:N] + 1)), 
AMWG(:Sigma, 1.0), 
Slice(:sigma2C, 10.0)] 

setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc (model, birats, inits, 10000, burnin-2500, thin=2, chains=2) 
describe (sim) 


Results 
Iterations = 2502:10000 
Thinning interval = 2 


Chains = 1,2 
Samples per chain = 3750 


Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
mu beta[1] 106.7046188 2.258246468 0.0260759841 0.081338282 770.81949 
mu beta[2] 6.1804557 0.104047928 0.0012014420 0.004102793 643.14317 
sigmaC 6.1431758 0.460583341 0.0053183583 0.021005830 480.76933 
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Quantiles: 
2.5$ 25.02 50.0% 75.0% 97.5% 
mu beta[1] 102.3595659 105.2252185 106.6914834 108.164852 111.2001520 
mu, beta[2] 5.9720904 6.1130035 6.1817455 6.248025 6.3848798 
sigmaC 5.3167022 5,8123935 6.1206859 6.445419 7.0971249 


Jaws: Repeated Measures Analysis of Variance 


An example from OpenBUGS /4// and Elston and Grizzle /22] concerning jaw bone heights measured repeatedly in 
a cohort of 20 boys at ages 8, 8.5, 9, and 9.5 years. 


Model Bone heights are modelled as 


yi; ~ Normal( X 8, 3) i — 1,...,20 
1 8 011 012 01,3 01,4 
X= 1 85 8- H S= 021 022 023 024 
03,1 93,2 033 034 
1 9.5 041 04,2 043 044 
Bo, 81  Normal(0, v 1000) 
Y ~ InverseWishart(4, I) 


where y; is a vector of the four repeated measurements for boy 1. In the model specification below, bone heights are 
arranged into a 1-dimensional vector on which a Block-Diagonal Multivariate Normal Distribution is specified. Also 
not that since X is a covariance matrix, it is symmetric withM » (M + 1) / 2 unique (upper or lower triangular) 
parameters, where M is the matrix dimension. 


Analysis Program 


using Mamba 


## Data 
jaws = Dict{Symbol, Any} ( 
¿Y => 
[47.8 48.8 49.0 49.7 
46.4 47.3 47.7 48.4 
46.3 46.8 47.8 48.5 
45.1 45.3 46.1 47.2 
47.6 48.5 48.9 49.3 
32:00 S852 5234/3: S3. v 
51.2 3340 54.9 54:5 
49.8 50.0 50.3 52.7 
48.1 50.8 52.3 54.4 
45.0 47.0 47.3 48.3 
51.2 51,4 51.6 515.9 
48.5 49.2 53.0 55.5 
5b2.1.52.8 53.7 55.0 
48.2 48.9 49.3 49.8 
49.6 50.4 51.2 51.8 
5057-5127 52.14 53.9 
47.2 47.7 48.4 49.5 
5343 54.6 55.1 55.3 
46.2 47.5 48.1 48.4 
46.3 47.6 51.3 51.8], 
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M = jaws[:M] = size(jaws[:Y], 2) 

N = jaws[:N] = Ei Xd. d) 

jaws[:y] = vec(jaws[:Y]' 

jaws[:x] = as, jaws [:age]) 


## Model Specification 
model = Model ( 


y = Stochastic(1l, 
(beta0, betal, x, Sigma) -> BDiagNormal (beta0 + betal +» x, Sigma), 
false 


beta0 = Stochastic( 
() -> Normal(0, sqrt(1000)) 


betal = Stochastic( 
() -> Normal (0, sqrt (1000) ) 


Sigma = Stochastic(2, 
M -> InverseWishart (4.0, eye(M) ) 


## Initial Values 

inits = [ 

Dict (:y => jaws[:y], :beta0 => 40, :betal => 1, :Sigma => eye(M)), 
Dict (:y => jaws[:y], :beta0 => 10, :betal => 10, :Sigma => eye(M)) 


## Sampling Scheme 

scheme = [Slice([:beta0, :betal], [10, 1]), 
AMWG(:Sigma, 0.1)] 

setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc(model, jaws, inits, 10000, burnin-2500, thin=2, chains=2) 
describe (sim) 


Results 
Iterations = 2502:10000 
Thinning interval = 2 


Chains = 1,2 
Samples per chain = 3750 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
Sigma[1,1] 6.7915801 2.0232463 0.0233624358 0.1421847433 202.48437 
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Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 


Quant 


Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 
Sigma 


sos 
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~ 


Gu € NO BO BÓ qd p [n 
sos 
SS 004 Y N BW PD 


~ 


Hs 
~ 
Hs 


beta0 3 


iles: 


sos > 


~ 


sos > 


~ 


WWN NO NO P2 P FP 
- 
Pwo PWN ds YN ES 


6.5982624 
6.1775526 
5.9477070 
6.9308723 
6.6005767 
6. 

7.4564163 
7.4518620 
8.0594440 
1.8742617 
366379701 


3803028 


Ez oC» BO NN BÓ qee BO qe Pe 


2.9% 


. 7202164 
.5674344 
.2043099 
.9143241 
.7936234 
.4721419 
2133129 
¿1213458 
.0756709 
.4482953 
.4349627 
.4960557 3 


.9670001 
.9084389 
.9358258 
.0236387 
.9885583 
.0196017 
.1925641 
.2712194 
.4746352 
-2272166 
499125039 


5 
5 
4 
4 
5 
Sa 
4 
5 
3 
6 
1 
2 


¡Mo e C; O OC O O O O CO 


25.0$ 
.3419070 
.2009878 
.8527075 
.5808041 
.4940524 
2183567 
.9659531 
,9139585 
;9961719 
.3090779 
.7279142 
.3922780 3 


.0227129612 
.0220367541 
40223529913 
.0233669666 
.0229618936 
.0233203515 
.0253175499 
. 0262257824 
. 0285746264 
0026236712 
0229929845 


50.0$ 
6.5046777 
6.3564397 
539476859 
5.6958961 
6.6730872 
6.3620683 
6.1310937 
7.1780478 
7.1240011 
7.6484712 
1.8707215 
3.6327451 3 


O'S OO OC. O e © 


7.8684049 
7.6419462 
7.1929746 
6.9962164 
8.0151463 
7.6617912 
Tes 

8.6551856 
8.7006336 
9.4043857 
2.0159440 2. 
4.9577696 


1469366529 
1532226770 


.1545185214 


1531630007 


.1600864592 


1612393116 
1705734202 
17334697137 
1784057891 


.0071954415 
.0632742554 


75.0% 


4443619 


179.20433 
155413523 
156.95367 
174.56542 
154.30055 
156.88795 
1655.22 733 
171.60713 
192.39975 
997,16079 
990, 37090 


97.5$ 


.5279247 
2 120602 
.8427648 
¿6203930 
.6796110 
.4419940 
.3714037 
.8617596 
.0597624 
.0451233 


3445976 


.4067853 


Eyes: Normal Mixture Model 


An example from OpenBUGS /47/ ], Bowmaker /6/, and Robert /73] concerning 48 peak sensitivity wavelength mea- 
surements taken on a set of monkey's eyes. 


Model Measurements are modelled as the mixture distribution 


y; ~ Normal(A7,, c) 
T; ~ Categorical( P) 
Ai ~ Normal(0, 1000) 
A2 =A1 +0 

0 ~ Uniform(0, 1000) 
c? ~ InverseGamma(0.001, 0.001) 
P = Dirichlet(1, 1) 


where y; is the measurement on monkey 2. 


Analysis Program 


using Mamba 


## Data 
eyes = 
:y => 
[529.0 
53544, 
5384295 


Dict{Symbol, Any} ( 


530.0, 
535.9, 
538.6, 


5320; 
53651; 
539.4, 


533.1; 
536.3, 
539.6, 


533.4, 
536.4, 
540.4, 


533.6, 
536.6, 
540.8, 


ess ly 
537.0, 
542.0, 


534.1; 
537.4, 
542.8, 


534.8, 
531759; 
543.0, 


535.3, 
538.3, 
543.5, 
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543.8, 543.9, 545.3, 546.2, 548.8, 548.7, 548.9, 549.0, 549.4, 549.9, 
550,6, 551.2, 551.42, 551.5, 551.6, 952.9, 5952.9, 553.2], 


:alpha => [1, 1] 


## Model Specification 
model - Model( 


y = Stochastic(1l, 
(lambda, T, s2, N) -> 
begin 
sigma = sqrt(s2) 
UnivariateDistribution[ 
begin 
mu = lambda[Int(T[i])] 
Normal (mu, sigma) 
end 
for i in 1:N 


T = Stochastic(l, 
(P, N) -» UnivariateDistribution[Categorical(P) for i in 1:N], 
false 

) YA 


P = Stochastic(1, 
alpha -» Dirichlet (alpha) 
), 


lambda = Logical(1, 
(lambda0, theta) -> Float64[lambda0; lambda0 + theta] 
), 


lambda0 = Stochastic( 
() => Normal(0.0, 1000.0), 
false 

), 


theta = Stochastic ( 
() -» Uniform(0.0, 1000.0), 
false 


s2 = Stochastic ( 
() -> InverseGamma(0.001, 0.001) 


## Initial Values 
inits = [ 
Dict (:y => eyes[:y], :T => fill(1, eyes[:N]), :P => [0.5, 0.5], 
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slambda0 => 535, :theta => 5, :s2 => 10), 
Dict (:y => eyes[:y], :T => fill(1, eyes[:N]), :P => [0.5, 0.5], 
:lambda0 => 550, :theta => 1, :s2 => 1) 
] 
## Sampling Scheme 
scheme = [DGS(:T), 
Slice([:lambda0, :theta], [5.0, 1.0]), 
Slice(:s2, 2.0, transform=true), 
SliceSimplex(:P, scale=0.75)] 
setsamplers! (model, scheme) 
## MCMC Simulations 
sim = mcmc (model, eyes, inits, 10000, burnin-2500, thin-2, chains=2) 
describe (sim) 
Results 
Iterations = 2502:10000 
Thinning interval = 2 
Chains = 1,2 
Samples per chain = 3750 
Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
P[1] 0.60357102 0.08379221 0.0009675491 0.0013077559 3750.00000 
P[2] 0.39642898 0.08379221 0.0009675491 0.0013077559 3750.00000 
S2 14.45234459 4.96689604 0.0573527753 0.1853970211  717.73584 
lambda[1] 536.75250337 0.88484533 0.0102173137 0.0304232594 845.90821 
lambda[2] 548.98693469 1.18938418 0.0137338256 0.0625489045 361.58042 
Quantiles: 
225% 25:0% 50.0$ 75.0% 97.5% 
P[1] 0.43576584 0.54842468 0.60395494 0.66066383 0.76552882 
P[2] 0.23447118 0233933617 0.39604506 0.45157532 0.56423416 
s2 8.55745263 11.37700782 13.39622105 16.26405571 27.08863166 
lambda[1] 535.08951002 536.16891895 536.73114152 537.30710397 538.61179506 
lambda[2] 546.57859195 548.24188698 548.97732892 549.74377421 551.38780148 


The BUGS Book 


Asthma: State Transitions in a Clinical Trial 


An example from the BUGS book /53/ concerning transitions between five clinical states in a randomized trial of 
treatments (seretide and fluticasone) for asthma. 


Model A discrete-time Markov model (equivalent to independent multinomial models) is fit with probability vector 
qi governing the state in the following week conditionally on the current state. Possible states are successfully treated, 
unsuccessfully treated, hospital-managed exacerbation, primary care-managed exacerbation, and treatment failure. 


The fifth state, treatment failure, is absorbing (patients cannot move out of it). The model is given by 
yij ~ Multinomial(M;, (qii, .. . i=1,...,3 


qi ~ Dirichlet(1,..., 1) 


; dis)) 
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where y;; is the number of transitions from state ¿ to 7 


Analysis Program 


using Mamba 


## Data 
asthma = Dict{Symbol, Any} ( 
:y => 
[210 600 1 1 
88 641 0 4 13 
1 000 1], 


## Model Specification 
model = Model ( 


y = Stochastic(2, 
(M, q) => 
MultivariateDistribution[ 
Multinomial(M[i], vec(q[i, :])) 
for i in 1:length (M) 
1s 


false 


q = Stochastic (2, 
M -> 
MultivariateDistribution[ 
Dirichlet (ones (5) ) 
for i in 1:length (M) 
] r 


true 


## Initial Values 
inits = [ 
Dict{Symbol, Any} ( 
fy => asthma[:y], 
:q => vcat([rand(Dirichlet(ones(5)))' for i in 1:3]...) 
) 


for i in 1:3 


## Sampling Scheme 
scheme = [SliceSimplex(:q)] 
setsamplers! (model, scheme) 


## MCMC Simulations 
sim = mcmc (model, asthma, inits, 10000, burnin-2500, thin-2, chains=3) 
describe (sim) 
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Results 
Iterations - 2502:10000 
Thinning interval = 2 
Chains = 1,2,3 
Samples per chain = 3750 
Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
q[1,1] 0.7615754849 0.0272201055 0.000256633616 0.001595328676 291.12484 
q[1,2] 0.2204851131 0.0265594084 0.000250404504 0.001578283076 283.18288 
q[1,3] 0.0034735444 0.0037556875 0.000035408962 0.000069146577 2950.10543 
q[1,4] 0.0072778962 0.0053705520 0.000050634050 0.000107382077 2501.34876 
q[1,5] 0.0071879614 0.0053672180 0.000050602617 0.000112487762 2276.60625 
q[2,1] 0.1191655126 0.0121038180 0.000114115890 0.000530342898 520.87229 
q[2,2] 0.8543825941 0.0130973639 0.000123483131 0.000564705907 537.92673 
q[2,3] 0.0012103544 0.0013675802 0.000012893670 0.000023654051 3342.67820 
q[2,4] 0.0066582050 0.0030978699 0.000029206998 0.000059253193 2733.39774 
q[2,5] 0.0185833339 0.0051526335 0.000048579495 0.000120299727 1834.54864 
q[3,1] 0.2936564126 0.1740764923 0.001641208908 0.005020530045 1202.21215 
q[3,2] 0.1394405572 0.1262073820 0.001189894609 0.002598964624 2358.13563 
q[3,3] 0.1424463856 0.1308387051 0.001233559141 0.002834635296 2130.48310 
q[3,4] 0.1417886606 0.1328770997 0.001252777310 0.003532010313 1415.32558 
q[3,5] 0.2826679840 0.1709210331 0.001611458954 0.004864906134 1234.36071 
Quantiles: 

2.5% 25.0% 50.0% 75.0% 97.5% 

q[1,1] 0.706775811059 0.74365894715 0.7617692483 0.7804747794 0.8125428018 
q[1,2] 0.169898760444 0.20150542266 0.2199140987 0.2386805795 0.2733246918 
q[1,3] 0.000059528767 0.00083424954 0.0022254753 0.0048346451 0.0138405267 
q[1,4] 0.000716682148 0.00327126033 0.0060441127 0.0099507177 0.0205431907 
q[1,5] 0.000740554613 0.00319616653 0.0058781978 0.0098161771 0.0209389022 
q[2,1] 0.096435484007 0.11060974070 0.1187015139 0.1271183450 0.1443574095 
q[2,2] 0.828193845237 0.84541382625 0.8550642143 0.8634806977 0.8792757738 
q[2,3] 0.000018088918 0.00028620290 0.0007503763 0.0016511135 0.0049571810 
q[2,4] 0.002033379513 0.00434074296 0.0061948558 0.0084704327 0.0138428435 
q[2,5] 0.010047515668 0.01486876182 0.0180566543 0.0217580995 0.0303924987 
q[3,1] 0.035105559792 0.15629556479 0.2700039867 0.4070428885 0.6826474093 
q[3,2] 0.002527526439 0.03939967503 0.1024828775 0.2077906692 0.4616490169 
q[3,3] 0.002612503802 0.03901018402 0.1041953555 0.2089759155 0.4719934399 
q[3,4] 0.002859691566 0.03827679857 0.1016367373 0.2072463302 0.4865686565 
q[3,5] 0.033209396793 0.14610521565 0.2577690743 0.3965462343 0.6524841408 


2.5.2 Contributed 


Additional examples are provided below to further illustrate features of the package. 


Line: Approximate Bayesian Computation 


A simple example to demonstrate the Approximate Bayesian Computation (ABC) sampler within the MCMC frame- 
work, based on the linear regression model defined in the Tutorial section. ABC sampling is applied separately to the 
:beta and : s2 parameter blocks. Different summary statistics are specified to show a range of functions that could 
be used. More common practice is to use the same data summaries for all ABC-sampled parameters. 
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Analysis Program 


using Mamba 


## Data 

line = Dict{Symbol, Any} ( 
ix e» [1,27 3, 4; 5]; 
¿y => [1, 3, 3, 3, 5] 


line[:xmat] = [ones(5) line[:x]] 


## Model Specification 
model = Model ( 
y = Stochastic(l1, 
(xmat, beta, s2) -> MvNormal(xmat * beta, sqrt(s2)), 
false 
) , 
beta = Stochastic(1, () -> MvNormal(2, sqrt(100))), 
s2 = Stochastic(() -> InverseGamma(0.01, 0.01)) 


## Initial Values 
inits = [ 
Dict{Symbol, Any} ( 
¿y => line[:yl, 
:beta => rand(Normal (0, 1), 2), 
:s2 => rand(Gamma (1, 1)) 
) 


for i in 1:3 


## Tuning Parameters 


scalel = [0.5, 0.25] 
summaryl - identity 
epsl = 0.5 


scale2 = 0.5 
summary2 = x -> [mean (x); sqrt (var (x))] 
eps2 = 0. 


H 


## User-Defined Sampling Scheme 


scheme = [ 
ABC(:beta, scalel, summaryl, epsl, kernel=Normal, maxdraw=100, nsim=3), 
ABC (:s2, scale2, summary2, eps2, kernel=Epanechnikov, maxdraw=100, nsim=3) 


] 


setsamplers! (model, scheme) 


## MCMC Simulation with Approximate Bayesian Computation 
sim = mcmc(model, line, inits, 10000, burnin-1000, chains=3) 
describe (sim) 
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Results 


Iterations - 1001:10000 
Thinning interval - 1 
Chains = 1,2,3 


Samples per chain 9000 


ll 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 
s2 1.30743333 1.99877929 0.0121641834 0.083739029 569.73624 
beta[1] 0.72349922 1.03842764 0.0063196694 0.039413390 694.16848 
beta[2] 0.77469344 0.31702542 0.0019293553 0.011392989 774.30630 


o 


Quantiles: 
24.55 25.05 50.0% 75.0% 97.5% 
s2 0.048095084 0.23351203 0.57947788 1.45858829 7.7639321 
beta[1] -1.309713807 0.12616636 0.67263204 1.27579373 3.1735176 
beta[2] 0.107216316 0.59367961 0.77867235 0.95156086 1.4043715 


Line: Block-Specific Sampling with AMWG and Slice 
An example based on the linear regression model defined in the Tutorial section. The program below illustrates use 


of the stand-alone amwg ! () and slice! () functions to sample different parameter blocks within the same MCMC 
algorithm. 


Analysis Program 


HAMAAARMAA MARA RARA RAR AA AARARARAA RAR ARAN AARANARAARARANARRARAR AR ARANA RARA AN RANA ARA 
## Linear Regression 

## y ~ N(b0 + bl * x, 82) 

## b0, bi ~ N(O, 1000) 

## s2 ~ invgamma(0.001, 0.001) 

HMMAAARAA AAA RARA DARAN AARARARAAAARANANAARARARAARARARAAAAR ARANA NAARANANANAANAN ARA 


using Mamba 
## Data 
data = Dict{Symbol, Any} ( 


:X => [1, 2, 3, 4, 5], 
¿y => [1, 3, 3, 3, 5] 


## Log-transformed unnormalized joint posterior for b0, bl, and log(s2) 


logf = function(x::DenseVector) 
b0 = x[1] 
b1 = x[2] 
logs2 = x[3] 
(-0.5 * length(data[:y]) - 0.001) * logs2 - 
(0.5 * sumabs2 (data[:y] - bO - bl x* data[:x]) + 0.001) / exp(logs2) - 


0.5 + b0^2 / 1000 - 0.5 + b1^2 / 1000 
end 


## Log-transformed unnormalized full conditional densities for the model 
## parameters beta and log(s2) defined below in the MCMC simulation 
logf beta(x) = logf([x; logs2]) 
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logf_logs2(x) = logf([beta; x]) 


## MCMC simulation 

n = 10000 

burnin = 1000 

sim = Chains(n, 3, names = ["b0", "bi", "s2"]) 
beta = AMWGVariate([0.0, 0.0]) 

logs2 = SliceVariate([0.0]) 

for i in lin 


amwg! (beta, [1.0, 1.0], logf beta, adapt = (i <= burnin)) 
slice! (logs2, [5.0], logf_logs2) 
sim[i, :, 1] = [beta; exp(logs2) ] 

end 


describe (sim) 


Results 


Iterations = 1:10000 
Thinning interval = 1 
Chains = 1 

Samples per chain = 10000 


Empirical Posterior Estimates: 

Mean SD Naive SE MCSE ESS 
bO 0.64401798 0.99315634 0.0099315634 0.060725564 267.4805 
bl 0.78985612 0.29790444 0.0029790444 0.017888106 277.3481 
s2 1.20785292 2.96033511 0.0296033511 0.062566344 2238.7222 


Quantiles: 

2.5% 25.0% 50.0% 75.0% 97.5% 
HO -1,33127385 0.075527035 026403226 1,.1902579 2.76655l7 
bl 0.16055439 0.625740352 0.7923275 0.9527029 1.3903861 
s2 0.16673189 0.381185645 0.6538295 1.2373814 5.6065938 


e 


Pollution: Bayesian Variable Selection 


Data from McDonald and Schwing /58/ that include 15 independent variables and a measure of mortality on 60 U.S. 
metropolitan areas in 1959-1961. Originally, the data were used to illustrate ridge regression (the full covariate matrix 
has a huge condition number). This dataset was included in a review of Bayesian variable selection techniques by 
O'Hara et al /64]. 
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Model 


y; ~ Normal(u;, c) i=1,...,60 
15 
wi = at) 0705 
j=l 
0; = BY 
a, Bj ~ Normal(0, 1000) 


"yj ~ Bernoulli(0.5) 
0” ~ InverseGamma(0.0001, 0.0001) 


Analysis Program 


using Mamba 


## Data 
data = [ 
36 27 TL. 861 3634 11.4 81.5 3243 8.8 42.6 11.7 21 15 59 59 3921.87 
35 23 72 11.1 3.14 11.0 78.8 4281 3.5 50.7 14.4 8 10 39 57 997.88 
44 29 74 10.4 3.21 29.8 81.6 4260 0.8 39.4 12.4 6 6 33 54 962.35 
47 45 79 6.5 3.41 11.1 77.5 3125 27.1 50.2 20.6 18 8 24 56 982.29 
43 35 77 7.6 3.44 9.6 84.6 6441 24.4 43.7 14.3 43 38 206 55 1071.29 
53 45 80 7.7.9.45 10.2. 66.8 3325 38.5 43.1 25.5 30 32 72 54 1030.38 
43 30 74 10.9 3.23 12.1 83.9 4679 3.5 49.2 11.3 21 32 62 56 934.70 
45 30 73 9.3 3.29 10.6 86.0 2140 5.3 40.4 10.5 6 4 4 56 $899.53 
36 24 70 9.0 3.31 10.5 83.2 6582 8.1 42.5 12.6 18 12 37 61 1001.90 
36 27 72. 9.5 3:36 10.7 79.3 4213 6,7 41.0 13.2 T2 T. 20. 59 912.35 
52 422 79 LT $899 9.6 694.2 2302 22.2 41:3 24.2 18 8. 27 56 1017:61 
33 26 76 8.6 3.20 10.9 83.4 6122 16.3 44.9 10.7 88 63 278 58 1024.89 
40 34 TT 9.2 3:21 10.2 77.0 4101 13.0 45.7 15.1 26 (26 146 57 970.47 
35 28 71 8.8 3.29 11.1 86.3 3042 14.7 44.6 11.4 31 21 64 60 985.95 
37 31 75 8.0 3.26 11.9 78.4 4259 13.1 49.6 13.9 23 9 15 58 958.84 
35 46 85 7.1 3.22 11.8 79.9 1441 14.8 51.2 16.1 1 1 1 54 $860.10 
36 30 75 7.5 3.35 11.4 81.9 4029 12.4 44.0 12.0 6 4 16 58 936.23 
15 30 73. 8.2 3.15 12.2 84.2 4824. 4,7] 53.1 12.7 17 8 28 38 871.77 
31 27 74 7.2 3.44 10.8 87.0 4834 15.8 43.5 13.6 52 35 124 59 959.22 
30 24 72. 6.5 3.53 10.8 79.5 3694 13.1 33.8 12.4 11 4 11 61 2941.18 
31 45 85 7.3 3.22 11.4 80.7 1844 11.5 48.1 18.5 1 1 1 53 891.71 
3l 24° 42. 9.0 3.97 10.9 82:28 3226 DeL 45.2 12.3 5 3 20 61 871,34 
42 X0 77 Gl 3645 10,42 71:8 22069 22.7 41.4 1955 8 3 5 53. 971:12 
43 27 72. 9.0 3.25 11.5 87.1 2909 à7.2 51.6 9.5 7 3 10 56 887.47 
46 55 84 5.6 3.35 11.4 79.7 2647 21.0 46.9 17.9 6 5 1 59 952.53 
39 29 75 8.7 3.23 11.4 78.6 4412 15.6 46.6 13.2 13 7 33 60 968.67 
35 31 81 .9.2 3.10 12.0 78.3 3202 12.6 48.6 13.9 7 4 4. 55. 919.73 
43 32 74 10.1 3.38 09.5 79.2 3214 2.9 43.7 12.0 11 7 32 54 844.05 
11 53 68 9.2 2.99 12.1 90.6 4700 7.8 48.9 12.3 648 319 130 47 861.83 
30 35 TL 8:3 3.97 94.9 TT. 4474 13.1 42.6 17.7 38 37 193 57 989.27 
50 42 82 7.3 3.49 10.4 72.5 3497 36.7 43.3 26.4 15 18 34 59 1006.49 
60 67 82 10.0 2.98 11.5 88.6 4657 13.5 47.3 22.4 3 1 1 60 861.44 
30 20 69 8.8 3.26 11.1 85.4 2934 5.8 44.0 9.4 33 23 125 64 929.15 
25 12 73. 9.2 3428 12.1 83.1 2095 212.0 51.9 39.8 20 11 26 58: 857.62 
45 40 80 8.3 3.32 10.1 70.3 2682 21.0 46.1 24.1 17 14 78 56 961.01 
AG 30 72 10.2 3.16 11:3 83.2 3327 8.8 45.3 12.2 4 3 8 58 923.23 
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54 54 81 7.4 3.36 29.7 72.8 3172 31.4 45.5 24.2 20 17 1 62 1113.16 
42 33 77 9.7 3.03 10.7 83.5 7462 11.3 48.7 12.4 41 26 108 58 994.65 
42 32 76 9.1 3.32 10.5 87.5 6092 17.5 45.3 13.2 29 32 161 54 1015.02 
36 29 72. 9.5 3.32 10.6 77.6 3437 8.1 45.5 13.8 45 59 263 .56. 991.29 
3T 38 67 11.3 2:99 12.0 81.5 3387 326 50.3 13.5 56 (21, 44 73. 893.99 
42 29 72 10.7 3.19 10.1 79.5 3508 2.2 38.8 15.7 6 4 18 56 938.50 
41 33. 77 11.2. 3.08 9.6 79.9 4843 2.7 38.6 14.1 11 11 $89 54 946.19 
44.39 78 8.2 3.32 11.0 79.9 3768 28.6 49.5 17.5 12 9 48 53 1025.50 
32.25 72 10.9 3.21 11.1 82.5 4355 5.0 46.4 10.8 7 4 18 60 874.28 
34 32 79. 9.83 3,23  9.T 76.8 5Ll60 17.2 45.1 15.5 31 15 68 57 3953.56 
10 55 70 7.3 3.11 12.1 88.9 3033 5.9 51.0 14.0 144 66 20 61 839.71 
18 48 63 9.2 2.92 12.2 87.7 4253 13.7 51.2 12.0 311 171 $6 Fi. 3911.70 
13 49 68 7.0 3.36 12.2 90.7 2702 3.0 51.9 9.7 105 32 3 “UL 790.73 
35 40 64 9.6 3402 12.2 82.5 3626 5.7 54.3 10.1 20 T. 20 72 899.26 
45 28 74 10.6 3.21 11.1 82.6 1883 3.4 41.9 12.3 5 4 20 56 904.16 
38 24 72 9.8 3.34 11.4 78.0 4923 3.8 50.5 11.1 8 5..25 61 950.67 
31 26 73. 92:9 3422. 10.7 81:3 3249. 9.5 43:98 19.65 LI T 28° 59 3972;:46 
40 23 71 11.3 3.28 10.3 73.8 1671 2.5 47.4 13.5 5 2 11 60 912.20 
4l 37 T8- 6.2 3:25 12.3 89:5 5308 25.9 59.7 10.3 65 28 102 52 967.90 
28 32 81. 7.0 3.27 12.1 81.0 3665 7.5 51.6 13.2 4 2 1 54 823.76 
425 33 To. T.T 3.39 11.3 82.2 3152 12.1 47.3 10.9 14 11 42 56 2003.50 
45 24 70 11.8 3.25 11.1 79.8 3678 1.0 44.8 14.0 7 3 8 56 895.70 
42 33 76 9.7 3.22 9.0 76.2 9699 4.8 42.2 14.5 8 8 49 54 911.82 
38 28 72 8.9 3.48 10.7 79.8 3451 11.7 37.5 13.0 14 13 39 58 954.44 
] 
pollution = Dict{Symbol, Any} ( 
:y => data[:, end], 
:X => mapslices(x -> x / sqrt(var(x)), data[:, 1:(end - 1)], 1), 
ip => size(data, 2) = 1 
) 
## Model Specification 
model = Model ( 
y = Stochastic(1, (mu, sigma2) -» MvNormal(mu, sqrt(sigma2)), false), 
mu = Logical(1, (alpha, X, theta) -> alpha + X « theta, false), 
alpha = Stochastic(() -» Normal(0, 1000)), 
theta = Logical(l, (beta, gamma) -> beta .* gamma), 
beta = Stochastic(1l, 
p -> UnivariateDistribution[Normal(0, 1000) for i in 1:p], 
false 
), 
gamma = Stochastic(1, () -> Bernoulli(0.5)), 
sigma2 = Stochastic(() -> InverseGamma (0.0001, 0.0001)) 


## Gibbs Sampler for alpha and beta 
Gibbs_alphabeta Sampler ([:alpha, 


:beta], 
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(alpha, beta, sigma2, X, gamma, y) -> 
begin 
alphabeta distr = [alpha.distr; beta.distr] 


alphabeta mean - map(mean, alphabeta distr) 
alphabeta invcov = spdiagm(map(d -» 1 / var(d), alphabeta distr)) 


M = [ones(y) X « spdiagm(gamma)] 
Sigma = inv(M' » M / sigma2 + alphabeta invcov) 
mu = Sigma « (M' x y / sigma2 + alphabeta invcov * alphabeta mean) 


alphabeta rand = rand(MvNormal (mu, Sigma)) 
Dict(:alpha => alphabeta rand[1], :beta => alphabeta rand[2:end]) 
end 


Gibbs sigma2 = Sampler([:sigma2], 
(mu, sigma2, y) -> 
begin 
a = length(y) / 2.0 + shape(sigma2.distr) 
b = sumabs2(y - mu) / 2.0 + scale(sigma2.distr) 
rand(InverseGamma(a, b)) 
end 


## Initial Values 
y = pollution[:y] 
X = pollution[:X] 
p = size(X, 2) 


inits = [ 
Dict (:y => y, :alpha => mean(y), :gamma => rand(0:1, p), 
:beta => inv(X' « X + eye(p)) * X' x y, :sigma2 => var(y)), 


Dict(:y => y, :alpha => 1, :gamma => rand(0:1, p), 

:beta => randn(p), :sigma2 => 1), 

Dict(:y => y, :alpha => 17, :gamma => rand(0:1, p), 

¿beta => [15, =15, -10, 5, -10, -5, =10, 10, 40, =5, 0, 0, 0, 20, 51, 

:sigma2 => 1), 

Dict(:y => y, :alpha => 17, :gamma => rand(0:1, p), 
beta => [15, I5, =10; 5; -—10,--5, —10, 10, 240, =5; 0, Oy 0, 20; Sly 
:Sigma2 => 1) 


## Sampling Scheme (without gamma) 
scheme0 = [Gibbs alphabeta, Gibbs sigma2] 


## Binary Hamiltonian Monte Carlo 

schemel = [BHMC(:gamma, (2 x p + 0.5) * pi); scheme0] 

setsamplers! (model, schemel) 

Siml = mcmc(model, pollution, inits, 10000, burnin=1000, thin-2, chains=4) 
describe (sim1) 


## Binary MCMC Model Composition 

scheme3 = [BMC3(:gamma); scheme0] 

setsamplers! (model, scheme3) 

sim3 = mcmc(model, pollution, inits, 10000, burnin-1000, thin-2, chains=4) 
describe (sim3) 


## Binary Metropolised Gibbs Sampling 
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scheme2 = [BMG(:gamma); scheme0] 

setsamplers! (model, scheme2) 

sim2 = mcmc(model, pollution, inits, 10000, burnin=1000, thin=2, chains=4) 
describe (sim2) 


## Discrete Gibbs Sampling 

scheme4 = [DGS(:gamma); scheme0] 

setsamplers! (model, scheme4) 

sim4 = mcmc(model, pollution, inits, 10000, burnin=1000, thin=2, chains=3) 
describe (sim4) 


Results 


## Binary Hamiltonian Monte Carlo 


Iterations = 1002:10000 
Thinning interval = 2 
Chains = 1,2,3,4 
Samples per chain 


4500 


Empirical Posterior Estimates: 
Mean SD Naive SE MCSE ESS 


gamma [1 0.494666667 0.49998544 0.00372667146 0.0354668606 198.73265 
gamma [2 0.147444444 0.35455827 0.00264272128 0.0244410034 210.44430 
gamma [3 0015611111 0.12396878 0.00092400872 0.0055061412 506.90895 
gamma [4 0.082888889 0.27572186 0.00205510941 20.0154493001 318.51125 
gamma [5 0.030722222 0.17256889 0.00128625256 0.0105253177 268.81567 
gamma [6 0.318833333 0.46603724 0.00347363646 0.0336986753 191.25622 
gamma [ 7 0.052777778 0.22359575 0.00166658436 0.0135283814 273.17152 
gamma [8 0.039055556 0.19373256 0.00144399723 0.0068660579 796.14158 
gamma [9 0.963111111 0.18849422 0.00140495300 0.0115061554 268.37102 
gamma [10 0.014388889 0.11909088 0.00088765098 0.0047463037 629.57277 
gamma [11 0.022888889 0.14955344 0.00111470550 0.0070783015 446.41078 
gamma [12 0.094222222 0.29214575 0.00217752582 0.0191631169 232.41643 
gamma [13 0.113000000 0.31660159 0.00235980895 0.0206601029 234.83414 
gamma [14 0.598888889 0.49013706 0.00365326592 0.0327788035 223.58820 
gamma [15 0. 008333333 0.09090846 0.00067759165 0.0023285399 1524.19744 
theta[l 10.963490356 11.99268806 0.08938821911 0.8074731491 220.58547 
theta[2 -3.052823000 7.78859258 0.05805274150 0.5173033385 226.68742 
theta[3 -0.180897422 1.69190595 0.01261072235 0.0669381806 638.85842 
theta[4 1.445892275 5.77285839 0.04302834596 0.3387376553 290.43895 
theta[5 0.220991524 1.67908086 0.01251512983 0.0755035072 494.54860 
theta[6 -6.639050625 10.52547301 0.07845224381 0.7272107620 209.48969 
theta[7 205695673210 2.97285383 0.02215834419 0.1667594094 317.80916 
theta[8 0.469028055 2.61485474 0.01948997649 0.0924855110 799.36980 
theta[9 32.862541601 12.24725603 0.09128565676 0.7176766659 291.21916 
theta[10 -0.120870758 1.29895656 0.00968185058 0.0429416368 915.02312 
theta[11 -0.111881745 3.05139958 0.02274378966 0.1397888103 476.48951 
theta[12 -11.124331528 40.10862414 0.29895203352 2.8217496939 202.04056 
theta[13 9.987999815 38.64436971 0.28803812538 2.7092141635 203.46316 
theta[14 14.017636096 12.48284270 0.09304161608 0.8185848401 232.54103 
theta[15 0.027299181 0.58218805 0.00433937350 0.0086425051 4500.00000 

sigma2 1675.673663690 421.46874551 3.14144255120 17.6104714349 572.78124 

alpha  937,545199598 187.23566554 1.39557225319 12.5267326120 223.40945 


Quantiles: 
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2.5% 25.0% 50.0% 75.0% 97.55 

gamma [1 0.000000 0.000000 0.000000 1.000000 1.0000000 
gamma [2 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [3 0.000000 0.000000 0.000000 0.000000 0.0000000 
gamma [4 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [5 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [6 0.000000 0.000000 0.000000 1.000000 1.0000000 
gamma [7 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [8 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [9 0.000000 1.000000 1.000000 1.000000 1.0000000 
gamma [10 0.000000 0.000000 0.000000 0.000000 0.0000000 
gamma [11 0.000000 0.000000 0.000000 0.000000 0.0000000 
gamma [12 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [13 0.000000 0.000000 0.000000 0.000000 1.0000000 
gamma [14 0.000000 0.000000 1.000000 1.000000 1.0000000 
gamma [15 0.000000 0.000000 0.000000 0.000000 0.0000000 
theta[l1 0.000000 0.000000 0.000000 22.137753 32.5989842 
theta[2 =21.095671 0.000000 0.000000 0.000000 0.0000000 
theta[3 0.000000 0.000000 0.000000 0.000000 0.0000000 
theta[4 0.000000 0.000000 0.000000 0.000000 23,6631537 
theta[5 0.000000 0.000000 0.000000 0.000000 2.2412885 
theta[6 -30.773041 -15.262916 0.000000 0.000000 0.0000000 
theta[7 =10'. 908665 0.000000 0.000000 0.000000 0.0000000 
theta[8 0.000000 0.000000 0.000000 0.000000 10,3679336 
theta[9 0.000000 25.808602 32.266120 39.686565 58.7865382 
theta[10 0.000000 0.000000 0.000000 0.000000 0.0000000 
theta[11 0.000000 0.000000 0.000000 0.000000 0.0000000 
theta[12] -163.621352 0.000000 0.000000 0.000000 0.0000000 
theta[13 -10.012373 0.000000 0.000000 0.000000 155.4851914 
theta[14 0.000000 0.000000 1727706561 24.764372 34.1579980 
theta[15 0.000000 0.000000 0.000000 0.000000 0.0000000 
sigma2 1074.425951 1385.703616 1605.131816 1878.631424 2759.1883162 
alpha 663.986294 798.280105 878.440305 1099.826940 1314.4802623 


2.6 Discussion 


Mamba is a platform for the development and application of MCMC simulators for Bayesian modelling. Such simu- 
lators can be difficult to implement in practice. Mamba eases that task by standardizing and automating the generation 
of initial values, specification of distributions, iteration of Gibbs steps, updating of parameters, and running of MCMC 
chains. It automatically evaluates (unnormalized) full conditionals and allows MCMC simulators to be implemented 
by simply stating relationships between data, parameters, and statistical distributions, similar to the ‘BUGS’ clones 
and Stan program. In general, the package is designed to give users access to all levels of MCMC design and imple- 
mentation. To that end, its toolset includes: 1) a model specification syntax, 2) stand-alone and integrated sampling 
functions, 3) a simulation engine and API, and 4) functions for convergence diagnostics and posterior inference. More- 
over, its tools are designed to be modular so that they can be combined, extended, and used in ways that best meet 
users’ needs. 


The package accommodates a wide range of model formulations as well as combinations of user-defined, supplied, and 
external samplers and distributions. It handles routine implementation tasks, thus allowing users to focus on design 
issues and making the package well-suited for 


* developing new Bayesian models, 
* implementing simulators for classes of models, 


* testing new sampling algorithms, 
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* prototyping MCMC schemes for software development, and 
* teaching MCMC methods. 


Furthermore, output is easily generated with the simulation engine in a standardized format that can be analyzed 
directly with supplied or user-defined tools for convergence diagnostics and inference. Use of the package is illustrated 
with several examples. Future plans include additional sampler implementations and optimizations, development of 
alternative model-specification interfaces, and automatic specification of sampling schemes. The software is freely 
available and licensed under the open-source MIT license. It is hoped that the package will foster MCMC methods 
developed by researchers in the field and make their methods more accessible to a broader scientific community. 


2.7 Supplement 


2.7.1 Bayesian Linear Regression Model 


The unnormalized posterior distribution was given for a Bayesian Linear Regression Model in the tutorial. Additional 
forms of that posterior are given in the following section. 


Log-Transformed Distribution and Gradient 


Let £ denote the logarithm of a density of interest up to a normalizing constant, and VC its gradient. Then, the 
following are obtained for the regression example parameters 3 and 0 = log(o?), for samplers, like NUTS, that can 
utilize both. 


L(8,0ly) = log(p(ylB, 0)p(8)p(0)) 


= Caf - o0 - cg (alu - X8 v - XP) + 9.) 


- 3(8— te)" X: (8 — pa) 


1 Ay XB) Ez (6 -— Hr) 
= exp{0} 
V£(8,0y) = |, "E07 zig (Hy — XB)" (y — XB) + Br) 


2.8 References 


2.9 Indices 


* genindex 
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